Tablet computer input management system and method
Through the dynamic adjustment technology of the tablet input management system, the input methods are identified and optimized, and the problems of difficulty in identifying input methods and insufficient keyboard adjustment in the existing technology are solved, and the input experience and accuracy are improved.
Patent Information
- Application Number
- CN202410437834.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-12
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-04-12
AI Technical Summary
The prior art is difficult to quickly and accurately identify input methods suitable for current application scenarios, and it is impossible to effectively adjust the local features of the keyboard to improve the user input experience.
In the tablet input management system, the rendering module, application management module and interaction sensing module are used to obtain user input and application semantic features, and dynamically adjust the display parameters of the sub-window based on user habit characteristics and preset rules to optimize the keyboard layout and input form.
It realizes the rapid identification and recommendation of input methods suitable for current application scenarios, improves the accuracy and experience quality of user input, and meets the input needs of different application scenarios.
Smart Images

Figure CN118244932B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a tablet computer input management system and method. Background Art
[0002] As the use of tablet computers continues to expand, users have a growing need to flexibly input data through a variety of input methods. In order to improve the user input experience, CN109783046B proposes an intelligent digital assistant in a multi-tasking environment, which can determine the user's intention based on the first voice input and context information. However, it only adjusts the size of the virtual keyboard as a whole, and does not consider how to adjust the local features of the keyboard, nor can it quickly identify the input method suitable for the current application scenario.
[0003] Therefore, it is hoped to provide a tablet computer input management system and method, which can help to quickly and accurately identify the input method desired by the user and make the keyboard layout more reasonable. Summary of the invention
[0004] One of the embodiments of the present specification provides a tablet computer input management system, the system includes a rendering module, an application management module, an interaction sensing module and a processor; the processor is configured to: continuously obtain user input through the interaction sensing module; in response to obtaining a sub-window opening instruction: obtain the foreground application of the current tablet computer from the application management module; obtain a first screen sequence within a preset time through the rendering module, and determine application semantic features of the foreground application based on the first screen sequence; determine a recommended display parameter set for the sub-window based on the application semantic features and user habit features; determine target display parameters in the recommended display parameter set based on preset rules; and generate display instructions based on the target display parameters to control the display module of the tablet computer to display the sub-window according to the target display parameters.
[0005] One of the embodiments of the present specification provides a tablet computer input management method, the method comprising: continuously obtaining user input through an interactive sensing module; in response to obtaining a sub-window opening instruction: obtaining the foreground application of the current tablet computer from an application management module; obtaining a first screen sequence within a preset time through a rendering module, and determining application semantic features of the foreground application based on the first screen sequence; determining a recommended display parameter set for the sub-window based on the application semantic features and user habit features; determining target display parameters in the recommended display parameter set based on preset rules; and generating display instructions based on the target display parameters to control the display module of the tablet computer to display the sub-window according to the target display parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:
[0007] Figure 1 is a module diagram of a tablet computer input management system according to some embodiments of this specification;
[0008] Figure 2 is an exemplary flow chart of a tablet computer input management method according to some embodiments of this specification;
[0009] Figure 3 is a schematic diagram of determining application semantic features according to some embodiments of this specification;
[0010] Figure 4 is a schematic diagram of determining recommended display parameters according to some embodiments of this specification; DETAILED DESCRIPTION
[0011] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.
[0012] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
[0013] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0014] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0015] Figure 1 1 is a module diagram of a tablet computer input management system according to some embodiments of this specification. Figure 1 As shown, the tablet computer input management system 100 may include a rendering module 110 , an application management module 120 , an interaction sensing module 130 , and a processor 140 .
[0016] The rendering module 110 is used to render and display pictures, and send the rendered pictures to the processor.
[0017] The application management module 120 is used to obtain the foreground application of the current tablet computer and send the foreground application to the processor.
[0018] The interaction sensing module 130 is used to obtain user input. In some embodiments, the user input may include finger touch screen and voice, etc. The interaction sensing module 130 may include an input acquisition program.
[0019] In some embodiments, the processor 140 can be configured to: continuously obtain user input through the interactive sensing module; in response to obtaining a sub-window opening instruction: obtain the foreground application of the current tablet computer from the application management module; obtain the first screen sequence within a preset time through the rendering module, and determine the application semantic characteristics of the foreground application based on the first screen sequence; determine the recommended display parameter set of the sub-window based on the application semantic characteristics and user habit characteristics; determine the target display parameters in the recommended display parameter set based on preset rules; generate display instructions based on the target display parameters to control the display module of the tablet computer to display the sub-window according to the target display parameters.
[0020] In some embodiments, the processor 140 may also be configured to: determine a color block distribution sequence based on the first picture sequence; and determine an application semantic feature based on the color block distribution sequence and the foreground application.
[0021] In some embodiments, the processor 140 may also be configured to determine application semantic features through a vector database based on the color block distribution sequence and the foreground application.
[0022] In some embodiments, the processor 140 may also be configured to: determine input form features based on application semantic features; and determine a recommended display parameter set based on application semantic features, input form features, and user habit features.
[0023] In some embodiments, the processor 140 can also be configured to: in response to the input form feature being a text string, determine the initial recommended parameter set of the sub-window based on the application semantic features, the input form features and the user habit features; determine the user demand features based on the application semantic features and the user habit features; based on the user demand features, adaptively adjust the interactive size of at least one initial recommended parameter in the initial recommended parameter set to determine the recommended display parameter set.
[0024] For more information about the processor 140, see Figure 2 and its related description.
[0025] In some embodiments, the tablet computer input management system further includes a display module, which is used to display screen content to the user, and may include, for example, a display screen of a tablet computer. In some embodiments, the tablet computer input management system may interact with the display module of the tablet computer. For example, an instruction may be sent to the display module to control the display of the display module.
[0026] In some embodiments, the tablet computer input management system further includes a storage module, and the storage module is used to store data, and may include, for example, a memory and the like.
[0027] Figure 2 200 is an exemplary flow chart of a tablet computer input management method according to some embodiments of this specification. In some embodiments, process 200 may be executed by a processor. Figure 2 As shown, process 200 includes the following steps:
[0028] Step 210: continuously obtain user input through the interactive sensing module.
[0029] User input refers to data input by the user through different channels. For example, the user input at least includes a sub-window opening instruction, and the user input may also include other voice data, text data, image data, etc.
[0030] In some embodiments, the processor can obtain user input through touch screen, voice, Bluetooth, etc.
[0031] Step 220 , in response to obtaining the sub-window opening instruction, obtaining the foreground application of the current tablet computer from the application management module.
[0032] The sub-window refers to a window that allows the user to input data. For example, the sub-window may be a soft keyboard, and the sub-window may include at least one of a text input module, a voice input module, or a picture input module.
[0033] Users can use a mouse, touch screen, or other devices to input data through the subwindow.
[0034] The sub-window opening instruction refers to an instruction for opening a sub-window.
[0035] The foreground application refers to the application that the user is currently using, such as music, games, document editing, etc.
[0036] In some embodiments, the processor may obtain a list of currently running application programs through background application data of the current tablet computer to determine the foreground application.
[0037] Step 230: obtaining a first picture sequence within a preset time through a rendering module, and determining application semantic features of the foreground application based on the first picture sequence.
[0038] The first picture sequence refers to a pixel sequence of a picture displayed by the foreground application of the current tablet computer, wherein the pixel sequence refers to a sequence composed of a series of pixel data.
[0039] The preset time refers to a pre-set time range within which a pixel sequence of a display screen needs to be acquired.
[0040] In some embodiments, the preset time can be set based on historical experience or actual needs, for example, the preset time can be 1 second. At this time, the processor can obtain the pixel sequence of the screen displayed by the foreground application within 1 second from the current moment, and use it as the first screen sequence.
[0041] Application semantic features refer to the scenario in which the application is located.
[0042] The scene in which the application is located refers to the current application page, which can be represented by a positive integer value. For example, the "Alipay" APP may include 100 sub-scenes (such as the reprint page, the Ant Forest page, the Yu'ebao page, etc.), then the scene in which the "Alipay" APP is located can be taken from 1 to 100, where each value corresponds to a sub-scene or sub-page in the "Alipay" APP. The value corresponding to each sub-scene can be set in advance.
[0043] In some cases, due to the need to protect personal information, the processor cannot directly obtain the scene in which the application is located. In this case, it is necessary to identify it through the first picture sequence.
[0044] In some embodiments, the processor can determine the scene in which the application is located based on the first screen sequence. For example, if the type of the foreground application is the "Alipay" APP, the "Alipay" APP is currently in the "sub-page for transferring money to someone" or the "sub-page of Ant Farm", etc.
[0045] In some embodiments, the processor may determine the application semantic features of the foreground application through a first preset table. The first preset table is constructed based on at least one first screen sequence of the application in the historical data and the corresponding application semantic features, and the application semantic features with the highest frequency of occurrence are used as the application semantic features of the application under the first screen sequence. The historical data may include at least one of the historical usage data of the application and the test data of the application operation.
[0046] In some embodiments, the processor may also determine a color block distribution sequence based on the first picture sequence, and determine an application semantic feature based on the color block distribution sequence and the foreground application. For more detailed descriptions, see Figure 3 and its related description.
[0047] Step 240: Determine a recommended display parameter set for the sub-window based on the application semantic features and the user habit features.
[0048] The user habit feature refers to the sub-window usage feature that the user is accustomed to. For example, the user habit feature may include the layout type and language type of the sub-window that the user is more inclined to use.
[0049] The layout type refers to the type of sub-window layout. Different layout types have different keyboard types and key layouts. For example, the layout type can include voice, no voice, left-hand mode, right-hand mode, nine keys or twenty-six keys, etc.
[0050] Language type refers to the category of languages used for communication, for example, language type can include Chinese, English or other languages.
[0051] In some embodiments, the processor may pre-set the layout type and language type, and the user may adjust the settings as needed.
[0052] In some embodiments, the processor may obtain the user habit characteristics based on the user settings in the historical usage data, or may determine the user habit characteristics through the settings input by the user.
[0053] The recommended display parameter set includes at least one recommended display parameter. The recommended display parameter refers to a sub-window related parameter recommended for use.
[0054] Each recommended display parameter includes at least one of the layout type of the sub-window, the language type, the position where the sub-window appears on the display module of the tablet computer, and the size parameters of the sub-window, wherein the size parameters of the sub-window include the length and width of the sub-window, and the interactive size of each virtual key, and the interactive size may include the size of the position where the virtual key is located.
[0055] In some embodiments, the processor may determine the recommended display parameter set in a variety of ways.
[0056] In some embodiments, the processor may perform matching in a first reference database based on the first vector to be matched to determine a recommended display parameter set. The first vector to be matched may be constructed based on application semantic features of the foreground application and user habit features.
[0057] The first reference database includes a plurality of standard vectors. In some embodiments, the processor may cluster the historical application semantic features, historical user habit features, and historical display parameters in the historical data, obtain at least one cluster center, and construct at least one standard vector based on the historical application semantic features and historical user habit features corresponding to the cluster center.
[0058] In some embodiments, the processor may determine a standard vector whose similarity with the first vector to be matched is greater than a first threshold as a reference vector, and determine the historical display parameters corresponding to the reference vector as a recommended display parameter set.
[0059] In some embodiments, the processor may determine the input form feature based on the application semantic feature, and determine the recommended display parameter set based on the application semantic feature, the input form feature and the user habit feature. For more detailed description, please refer to this specification. Figure 4 Related description in .
[0060] In some embodiments, the processor may also determine user demand characteristics based on application semantic characteristics and user habit characteristics, adaptively adjust the initial recommended parameter set according to the user demand characteristics, and determine the recommended display parameter set. For more information about user demand characteristics, initial recommended parameter sets, and determining recommended display parameter sets, see Figure 4 and its related description.
[0061] Step 250: Determine target display parameters in the recommended display parameter set based on preset rules.
[0062] The target display parameter refers to a parameter used to display the sub-window. For example, the target display parameter may be a display parameter in a recommended display parameter set that best meets the user's needs.
[0063] In some embodiments, the processor may display a plurality of recommended display parameters in the recommended display parameter set to the user in the form of thumbnails or names, allowing the user to select a target display parameter.
[0064] In some embodiments, the processor may automatically select the display parameter with the highest recommendation value from the recommended display parameter set as the target display parameter. In some embodiments, the processor may determine the aforementioned recommended value in a variety of ways. For example, the higher the number of historical user selections, the higher the recommended value corresponding to the recommended display parameter. For another example, based on the recommended display parameter and the user habit characteristics corresponding to the current user, the recommended value corresponding to the recommended display parameter is determined by a pre-set algorithm, etc. Among them, the algorithm may be a machine learning algorithm or other possible algorithms.
[0065] In some embodiments, the processor may generate a display instruction based on the target display parameter to control the display module of the tablet computer to display the sub-window according to the target display parameter. The display instruction refers to an instruction for controlling the tablet computer to display the sub-window.
[0066] It should be noted that the above description of the relevant process is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the process under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.
[0067] Figure 3 This is a schematic diagram of determining application semantic features according to some embodiments of this specification.
[0068] In some embodiments, the processor may further determine a color block distribution sequence 320 based on the first picture sequence 310 , and determine an application semantic feature 350 based on the color block distribution sequence 320 and the foreground application 330 .
[0069] The color block distribution sequence 320 refers to a sequence composed of color block distribution within a preset time period, wherein the color block distribution refers to the position distribution of the color blocks on the screen.
[0070] A color block is an area with the same color characteristics. In some embodiments, the color block can be determined by a standard color and a threshold range.
[0071] The UI design of an application often controls the number of color blocks used within a limited range for aesthetic reasons, that is, one color is used as the theme color. By dividing the color blocks, the processor's computational workload is greatly reduced, which helps to improve processing efficiency.
[0072] The standard color is a preset theme color, and the threshold range is a preset color deviation range. In some embodiments, the standard color can be represented by the RGB value of the color.
[0073] As an example only, the processor may preset a standard color RGB value as (255, 0, 0), the threshold range may be set to ±20 of the standard color, and pixels with RGB values in the range of (235-275, 0, 0) may be regarded as a color block. In some embodiments, pixels that are not within the standard color threshold range may be recorded as a color block.
[0074] In some embodiments, the color block distribution can be determined based on the color block and the position of the color block on the screen. In some embodiments, the color block distribution can be represented in the form of a vector. For example, the vector Q(P 1 ,…,P m ) can be used to represent a color block distribution, where m represents a total of m color blocks, P m Indicates the features corresponding to the mth color block, including the color value corresponding to the color block m and the coordinates corresponding to multiple ranges of color m in the picture.
[0075] As an example only, P m Can be (RGB m ,(X 1 , Y 1 ),…,(X n , Y n )), where RGB m Indicates the color value corresponding to the color block m in the picture, (X n , Y n ) represents the coordinates corresponding to the nth range among multiple ranges of color m.
[0076] In some embodiments, the processor may count the color block distribution of each picture within a preset time period, and determine the color block distribution of multiple pictures as a color block distribution sequence 320 .
[0077] In some embodiments, the processor may determine the threshold range corresponding to the aforementioned standard color and the color block based on the foreground application 330 of the tablet computer.
[0078] In some embodiments, the processor may obtain style parameters of the foreground application 330 and obtain standard colors therefrom. The style parameters are parameters preset by the application and may include standard colors and the like.
[0079] In some embodiments, the threshold range can be set based on historical data or actual needs. For example, the color value difference from the standard color is no more than 10% as the threshold range.
[0080] For example, each application can preset its own standard color, and users can also change the application theme color.
[0081] In some embodiments of the present specification, by making the standard color and the threshold range related to the foreground application, the determination of the color block can be better adapted to applications of different styles.
[0082] In some embodiments, the processor may determine the application semantic characteristics of the foreground application in a variety of ways.
[0083] In some embodiments, the processor may query a preset table corresponding to the foreground application based on the current color block distribution of the image closest to the current moment in the selected color block distribution sequence 340, determine the reference color block distribution in the table that is most similar to the current color block distribution, and determine the application scenario corresponding to the reference color block distribution that is most similar to the current color block distribution as the application semantic feature 360. For more information about application scenarios and application semantic features, see Figure 2 and its related description.
[0084] In some embodiments, when the color block distribution corresponding to the current application is different from the vector length of the reference color block distribution in the preset table, the processor may align the lengths of the two by using a placeholder.
[0085] In some embodiments, the preset table can be constructed based on the scenarios of the foreground application. For example, for the WeChat application, there may be a chat dialogue interface, a circle of friends interface, and a mini-program interface, etc. The processor can construct the color block distribution corresponding to these application scenarios as a reference color block distribution and include it in the aforementioned preset table.
[0086] In some embodiments, the processor may also determine the application semantic feature 350 through the vector database 340 based on the color block distribution sequence 320 and the foreground application 330 .
[0087] In some embodiments, the processor may weight and fuse the color block distribution of each frame in the color block distribution sequence 320 in chronological order to obtain a fused global color block distribution; construct a second vector to be matched based on the global color block distribution and the foreground application 310, calculate the similarity between the vector to be matched and the second standard vector in the vector database 350, use the second standard vector with the highest similarity as the target vector, and determine the reference application semantic feature corresponding to the target vector as the application semantic feature 350 of the foreground application. The vector similarity may be calculated by Euclidean distance, normal distance or similar mathematical means.
[0088] In some embodiments, the processor may convert the color block distribution of each frame into a numerical vector, where the elements of the vector include color values corresponding to each color block arranged in a preset order.
[0089] In some embodiments, the weight in weighted fusion can be calculated based on the interval between the time when the color block distribution is displayed and the current time. The smaller the interval from the current time, the greater the weight corresponding to the color block distribution.
[0090] In some embodiments, the processor can determine historical semantic features based on historical applications and historical first screen sequences in the historical usage data of the tablet computer; perform clustering based on historical applications, historical color block distribution sequences and historical semantic features to determine at least one cluster center, and construct at least one second standard vector based on the historical applications and historical color block distribution sequences corresponding to each of the at least one cluster center, and include it in the vector database 340, and use the historical semantic features corresponding to the second standard vector as reference application semantic features.
[0091] In some embodiments, the processor may preset the number of cluster centers, and cluster the vectors composed of the historical applications, the historical color block distribution sequences, and the historical semantic features through a vector clustering algorithm to obtain cluster centers.
[0092] Historical applications refer to applications used on the tablet computer within a preset historical period.
[0093] The historical first screen sequence refers to the first screen sequence corresponding to the historical application 371 .
[0094] In some embodiments, historical applications can be obtained from historical data of the tablet computer; historical first screen sequences can be obtained from screen recording data. When screen recording data cannot be obtained, it can also be obtained through forced authorization, obtaining third-party camera images, etc.
[0095] The historical color block distribution sequence refers to the color block distribution sequence corresponding to the historical application. In some embodiments, the historical color block distribution sequence can be obtained based on the historical first picture sequence, and the acquisition method can be similar to the method of obtaining the color block distribution sequence. For more information on obtaining the color block distribution sequence based on the first picture sequence, see Figure 3 and its related description.
[0096] Historical semantic features refer to application semantic features corresponding to historical applications.
[0097] In some embodiments, the processor may extract key pictures from the historical first picture sequence, and determine the historical semantic features based on the key pictures, wherein the key pictures refer to pictures in the picture sequence that can represent the features of the picture sequence.
[0098] In some embodiments, historical semantic features can be manually annotated based on pixel information in key images and foreground applications, or can be annotated by querying a preset table. For more information about the preset table, see Figure 3 and its related description.
[0099] In some embodiments, the processor may calculate the similarity between consecutive frames, and randomly select a frame from the frames with a similarity higher than a preset threshold as a key frame. For example, for 80 consecutive frames with a similarity greater than 90%, one frame may be randomly selected as the key frame for this segment of consecutive frames, wherein the similarity may be calculated based on the difference in the RGB values of the pixels in the two frames.
[0100] In some embodiments, the historical semantic features may be determined based on the historical applications and the historical first screen sequences through a first determination model, wherein the first determination model may be a machine learning model, for example, the first determination model may be a deep neural network (DNN).
[0101] In some embodiments, the input of the first determination model may include historical applications and historical first screen sequences, and the output may include historical semantic features.
[0102] In some embodiments, the first determination model can be obtained by training based on multiple labeled training samples. For example, multiple labeled training samples can be input into the initial first determination model, and a loss function can be constructed by using the labels and the results of the initial first determination model. The parameters of the initial first determination model can be iteratively updated based on the loss function by gradient descent or other methods. When the preset conditions are met, the model training is completed and the trained first determination model is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.
[0103] In some embodiments, the training samples may include historical applications and historical first screen sequences in historical data.
[0104] In some embodiments, the label corresponding to the training sample may include the application semantic features actually corresponding to the training sample. The label may be manually annotated according to the actual application or may be annotated based on a preset table. For more information about the preset table, see Figure 3 and its related description.
[0105] In some embodiments of the present specification, a machine learning model is used to determine the historical semantic features corresponding to the second standard vector, and a vector database is used when actually determining the application semantic features, thereby ensuring the response time of the foreground application.
[0106] In some embodiments of the present specification, matching is performed based on color blocks, which can reduce the amount of calculation. Color blocks are high-order features extracted from pixels, contain less information and are sufficient for identification, which can avoid excessive calculation of the processor and improve processing efficiency.
[0107] Figure 4 It is a schematic diagram of determining a recommended display parameter set according to some embodiments of this specification.
[0108] In some embodiments, the processor may determine the input form feature 420 based on the application semantic feature 410 ; and determine the recommended display parameter set 460 based on the application semantic feature 410 , the input form feature 420 , and the user habit feature 430 .
[0109] The input form feature 420 refers to the form feature of the user's high-frequency input in the current foreground application. For example, the input form feature can be a number, a text string, or a voice.
[0110] In some embodiments, the input form feature 420 can be determined based on user information, foreground application, and application semantic features by querying a second preset table, wherein the second preset table may include reference user information, sample foreground application, reference application semantic features, and corresponding reference input form features.
[0111] In some embodiments, the second preset table can be constructed in advance according to basic information of the scenario and application.
[0112] In some embodiments, the processor may perform a match in the second reference database based on the third vector to be matched, and determine a recommended display parameter set.
[0113] The third vector to be matched may be constructed based on current user information, current foreground application, current application semantic features, current input form features, and current user habit features.
[0114] The second reference database includes a plurality of reference vectors and their corresponding reference display parameter sets. In some embodiments, the second reference database may be constructed based on historical usage data of the tablet computer.
[0115] In some embodiments, the processor may calculate the distance between the reference vector and the third vector to be matched, and determine the recommended display parameter set corresponding to the third vector to be matched. For example, a reference vector whose distance from the third vector to be matched satisfies a preset condition is used as a target vector, and the reference display parameter corresponding to the target vector is used as the recommended display parameter set corresponding to the third vector to be matched. The preset condition may be set according to the situation. For example, the preset condition may include selecting a reference vector whose vector distance is less than a distance threshold, selecting N reference vectors whose vector distance is the smallest, etc.
[0116] In some embodiments, in response to the input form feature 420 being a text string, the processor may determine an initial recommended parameter set 440 for the sub-window based on the application semantic feature 410, the input form feature 420, and the user habit feature 430; determine a user demand feature 450 based on the application semantic feature 410 and the user habit feature 430; and adaptively adjust the interactive size of at least one initial recommended parameter in the initial recommended parameter set 440 based on the user demand feature 450 to determine a recommended display parameter set 460.
[0117] For more information about application semantic features, user habit features, and recommended display parameter sets, see Figure 2 and its related description.
[0118] In some embodiments, the processor may determine the initial recommended parameter set 440 based on the application semantic features of the foreground application and the user habit features. The processor may also determine the initial recommended parameter set 440 based on user information, foreground application, application semantic features, input form features, and user habit features. The specific details of determining the initial recommended parameter set 440 are similar to determining the recommended display parameter set, and can be found in this specification. Figure 2 and Figure 4 See the relevant instructions in .
[0119] User demand feature 450 is information related to the text that the user may input in the current interface. For example, user demand feature 450 may include at least one of the specific text that the user is going to input, the frequency of use of each spelling unit that the user is going to input, or a set of high-frequency spelling units that the user is going to input.
[0120] The specific text refers to an input text string, for example, the specific text may be "Hello, Goodbye".
[0121] The usage frequency of each spelling unit refers to the number of times each spelling unit corresponding to the input text appears. For different input methods, the type of spelling unit may be different. For example, for a pinyin-based input method, the type of spelling unit is each individual English character, such as a, b, c, etc. For a Wubi-based input method, the type of spelling unit includes each individual English character (such as a, b, c, etc.) and the corresponding Wubi radical of each Chinese character.
[0122] A high-frequency spelling unit refers to a spelling unit whose usage frequency is greater than a first frequency threshold. In some embodiments, the processor may count the occurrence frequency of each spelling unit input by the user in history, and take a spelling unit whose usage frequency is greater than the first frequency threshold in the text to be input by the user in the text box displayed on the tablet computer as a high-frequency spelling unit, and form all high-frequency spelling units into a high-frequency spelling unit set.
[0123] In some embodiments, the processor obtains the historical text actually input by the user corresponding to the application semantic features and user habit features of the current foreground application in the historical data, and uses the relevant information of the actually input historical text as the user demand features of the current foreground application, wherein the relevant information includes the specific text, the frequency of use of each spelling unit or a set of high-frequency spelling units.
[0124] For example, in a certain application scenario under a certain application, if the layout type of the sub-window is nine-key, and the high-frequency spelling units input by the user in history are spelling units involved in the text "hello, goodbye", then the corresponding user demand features are spelling units involved in the text "hello, goodbye".
[0125] In some embodiments, the processor may also determine the user demand characteristics through a second determination model based on application semantic characteristics, user habit characteristics, user behavior sequence within a preset time, and application usage sequence. For more information about application semantic characteristics, user habit characteristics, and preset time, see Figure 2 and its related description.
[0126] The user behavior sequence refers to the sequence of user touch screen operations arranged in time. For example, the user behavior sequence may include "user touches key a", "user touches key b", "user touches voice input", etc.
[0127] The application usage sequence refers to the sequence of applications opened by the user arranged in time. For example, the application usage sequence may include "Alipay", "Taobao", "WeChat", etc.
[0128] In some embodiments, the second determination model is a machine learning model. For example, the second determination model may be a recurrent neural network model (RNN).
[0129] The input of the second determination model may include application semantic features, user habit features, user behavior sequences and application usage sequences within a preset time, and the output of the second determination model may include user demand features.
[0130] In some embodiments, the input of the second determination model also includes a color patch distribution sequence.
[0131] For more information about the color block distribution sequence, see Figure 3 and its related description.
[0132] In some embodiments, the training samples used to train the second determination model can be sample application semantic features, sample user habit features, sample behavior sequences of users within a preset time, and sample application usage sequences in the historical data within the first historical time. The training samples can be determined based on the historical data.
[0133] For each training sample, the text string actually input by the corresponding user at the second historical time is counted to obtain the usage frequency of each character involved in the text string, and the characters that meet the preset selection conditions are combined into a character set, and the character set is determined as the corresponding user demand feature as the label of the training sample. The second historical time is later than the first historical time.
[0134] The second frequency threshold can be set according to actual conditions; the preset selection conditions may include that the usage frequency of the spelling unit is greater than the second frequency threshold, or the usage frequency of the spelling unit is ranked high from high to low (such as the top N, etc.).
[0135] In some embodiments, the processor may adaptively adjust at least one initial recommended parameter in the initial recommended parameter set 440 based on the user demand feature 450 through a preset algorithm to determine the recommended display parameter set 460 .
[0136] Among them, adaptively adjusting at least one initial recommended parameter includes adaptively adjusting the display effect of the virtual key in at least one initial recommended parameter, and the display effect of the virtual key includes interactive size, color type, font thickness, etc.
[0137] Preset algorithms may include the following:
[0138] In some embodiments, the processor can determine the language type and layout type that the user prefers based on the user's habit characteristics. For example, the language type and layout type that the user prefers are directly obtained based on the user's habit characteristics. For detailed descriptions of the user's habit characteristics, please refer to this specification. Figure 2 Related description in .
[0139] In some embodiments, the processor may also determine the initial recommended parameter set based on the application semantic features of the foreground application. For example, in response to being able to directly obtain the application scenario of the foreground application, the processor may retrieve at least one display parameter from the preset display parameters corresponding to the aforementioned application scenario based on the application semantic features of the foreground application as the initial recommended parameter set.
[0140] In some embodiments, the processor may adaptively adjust at least one display parameter in the initial recommended display parameter set according to user demand characteristics, thereby updating the initial recommended display parameter set.
[0141] In some embodiments, the processor may adaptively adjust the target parameter in the initial recommendation display parameter set, where the target parameter may include each recommendation parameter in the initial recommendation parameter set 440 or at least one initial recommendation parameter selected from the initial recommendation parameter set 440 according to the language type in the user habit feature 430 .
[0142] In some embodiments, the processor may determine the estimated interaction frequency of each virtual key in the initial recommended parameters to be adaptively adjusted according to the user demand characteristics and the layout type of the sub-window in the initial recommended parameters to be adaptively adjusted, wherein the estimated interaction frequency refers to the number of times each virtual key is expected to be clicked.
[0143] For example, if the user demand feature is the specific text "Haha", the layout type of the subwindow in the initial recommended parameters to be adaptively adjusted is a 26-key pinyin soft keyboard. To complete the input of the text "Haha", you need to click the virtual keys "h", "a", "h", "a", "space" in sequence. At this time, the estimated interaction frequency of the virtual key "h" is 2, the estimated interaction frequency of the virtual key "a" is 2, the estimated interaction frequency of the virtual key "space" is 1, and the estimated interaction frequency of other virtual keys is 0. If the input method is the Wubi input method, at this time, each virtual key corresponds to each stroke or radical of each Chinese character, and the processor can determine the estimated interaction frequency according to the Chinese character writing rules and the rules of the Wubi keyboard.
[0144] In some embodiments, the processor may adjust the display effect of each virtual key according to the estimated interaction frequency of each virtual key. Wherein, adjusting the display effect of each virtual key may include adjusting the interaction size, color or font thickness of each virtual key.
[0145] For example, adjusting the display effect may include enhancing the display effect, weakening the display effect, or maintaining the display effect. Among them, enhancing the display effect may include increasing the interactive size, changing the color, or making the font bolder, etc. Weakening the display effect may include reducing the interactive size, changing the color, or making the font thinner, etc. Maintaining the display effect means not making any adjustment to the current display effect.
[0146] In some embodiments, for virtual keys whose estimated interaction frequency is greater than a third frequency threshold, the display effect can be enhanced; for virtual keys whose estimated interaction frequency is less than a fourth frequency threshold, the display effect can be weakened; for virtual keys whose estimated interaction frequency is between the third frequency threshold and the fourth frequency threshold, the current display effect can be maintained.
[0147] In some embodiments, the increase in the interaction size of the virtual key may be positively correlated to its corresponding estimated interaction frequency; and the decrease in the interaction size of the virtual key may be negatively correlated to its corresponding estimated interaction frequency.
[0148] In some embodiments, the third frequency threshold and the fourth frequency threshold can be determined empirically, wherein the third frequency threshold is greater than the fourth frequency threshold.
[0149] In some embodiments, each virtual key may have a different third frequency threshold and fourth frequency threshold, and the third frequency threshold and fourth frequency threshold of each virtual key are related to the historical false touch frequency of the virtual key. For example, the higher the historical false touch frequency, the smaller the third frequency threshold of the virtual key, and the larger the fourth frequency threshold. The third frequency threshold is greater than the fourth frequency threshold.
[0150] In some embodiments, the historical false touch frequency is positively correlated with the number of historical false touches. In some embodiments, whether it is a false touch can be determined by the distance between the touch position and the boundary of the virtual key. For example, if the touch position is far away from the boundary of the virtual key, it is determined that it may be a false touch.
[0151] In some embodiments, the processor may also adaptively adjust the response sensitivity of each virtual key in the initial recommended parameter set according to the estimated interaction frequency of each virtual key.
[0152] Responsiveness refers to the speed and accuracy with which the processor responds to a user touching or clicking a virtual button.
[0153] In some embodiments, the adjustment of the reaction sensitivity is similar to the method of adjusting the display effect of the virtual key described above. For virtual keys whose estimated interaction frequency is greater than the fifth frequency threshold, the reaction sensitivity can be enhanced; for virtual keys whose estimated interaction frequency is less than the sixth frequency threshold, the reaction sensitivity can be weakened; for virtual keys whose estimated interaction frequency is between the fifth frequency threshold and the sixth frequency threshold, the reaction sensitivity may not be adjusted.
[0154] In some embodiments, each virtual key may have a different fifth frequency threshold and sixth frequency threshold, and the fifth frequency threshold and sixth frequency threshold of each virtual key are related to the historical false touch frequency of the virtual key. For example, the higher the historical false touch frequency, the larger the fifth frequency threshold and the sixth frequency threshold of the virtual key. The fifth frequency threshold is greater than the sixth frequency threshold.
[0155] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.
[0156] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.
[0157] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.
[0158] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.
[0159] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values is as accurate as possible within the feasible range.
[0160] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.
[0161] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.
Claims
1. A tablet computer input management system, characterized in that: It includes a rendering module, an application management module, an interactive sensing module and a processor; The processor is configured to: Continuously obtaining user input through the interactive sensing module; In response to obtaining the sub-window opening instruction: Get the foreground application of the current tablet computer from the application management module; Acquire a first picture sequence within a preset time through the rendering module, and determine an application semantic feature of the foreground application through a first preset table based on the first picture sequence, where the application semantic feature is a scene in which the application is located; The first preset table is constructed based on at least one first screen sequence of applications in historical data and the corresponding application semantic features, and the application semantic features with the highest frequency of occurrence are used as the application semantic features of the application under the first screen sequence; The application semantic features of the foreground application are further determined by a preset table corresponding to the foreground application, including: Based on the first picture sequence, determining a color block distribution sequence; Based on selecting the current color block distribution of a picture closest to the current moment in the color block distribution sequence, querying the preset table corresponding to the foreground application, determining the reference color block distribution in the preset table that is most similar to the current color block distribution, and determining the application scenario corresponding to the reference color block distribution that is most similar to the current color block distribution as the application semantic feature; Based on the application semantic features, user information and the foreground application, an input form feature is determined through a second preset table, wherein the input form feature is a form feature of a high-frequency input by the user in the foreground application, and the second preset table includes reference user information, sample foreground applications, reference application semantic features and corresponding reference input form features; Determining a recommended display parameter set of a sub-window based on the application semantic features, the input form features and the user habit features, the sub-window being a soft keyboard that allows the user to input data; The recommended display parameter set includes at least one recommended display parameter; Determining the recommended display parameter set of the sub-window further includes: In response to the input form feature being a text string, determining an initial recommended parameter set for the sub-window based on the application semantic feature, the input form feature and the user habit feature, the initial recommended parameter set including at least one initial recommended parameter; Based on the application semantic features, the user habit features, the color block distribution sequence, and the user behavior sequence and application usage sequence within a preset time, determining the user demand features through a second determination model, wherein the user demand features are related information reflecting the text that the user may input in the current interface, and the second determination model is a machine learning model; Determining an estimated interaction frequency of each virtual key in the initial recommended parameter to be adaptively adjusted based on the user demand characteristics and the layout type of the sub-window in the initial recommended parameter to be adaptively adjusted; Adaptively adjusting the interaction size and reaction sensitivity of each virtual key based on the estimated interaction frequency to determine the recommended display parameter set; Determining the recommended display parameter set of the sub-window further includes: Constructing a third to-be-matched vector based on the user information, the foreground application, the application semantic feature, the input form feature, and the user habit feature; Based on the third vector to be matched, matching is performed in a second reference database to determine the recommended display parameter set; Determining a target display parameter in the recommended display parameter set based on a preset rule; A display instruction is generated based on the target display parameter to control a display module of the tablet computer to display the sub-window according to the target display parameter.
2. The tablet computer input management system according to claim 1, characterized in that: The processor is further configured to: Based on the color block distribution sequence and the foreground application, the application semantic feature is determined through a vector database.
3. A tablet computer input management method, characterized in that: The method is executed based on a processor and includes: Continuously obtain user input through the interactive sensing module; In response to obtaining the sub-window opening instruction: Get the foreground application of the current tablet computer from the application management module; Obtaining a first screen sequence within a preset time through a rendering module, and determining an application semantic feature of the foreground application through a first preset table based on the first screen sequence, wherein the application semantic feature is a scene in which the application is located; the first preset table is constructed based on at least one first screen sequence of applications in historical data and the corresponding application semantic feature, and the application semantic feature with the highest frequency of occurrence is used as the application semantic feature of the application under the first screen sequence; The application semantic features of the foreground application are further determined by a preset table corresponding to the foreground application, including: Based on the first picture sequence, determining a color block distribution sequence; Based on selecting the current color block distribution of a picture closest to the current moment in the color block distribution sequence, querying the preset table corresponding to the foreground application, determining the reference color block distribution in the preset table that is most similar to the current color block distribution, and determining the application scenario corresponding to the reference color block distribution that is most similar to the current color block distribution as the application semantic feature; Based on the application semantic features, user information and the foreground application, an input form feature is determined through a second preset table, wherein the input form feature is a form feature of a high-frequency input by the user in the foreground application, and the second preset table includes reference user information, sample foreground applications, reference application semantic features and corresponding reference input form features; Determining a recommended display parameter set of a sub-window based on the application semantic features, the input form features and the user habit features, the sub-window being a soft keyboard that allows the user to input data; The recommended display parameter set includes at least one recommended display parameter; Determining the recommended display parameter set of the sub-window further includes: In response to the input form feature being a text string, determining an initial recommended parameter set for the sub-window based on the application semantic feature, the input form feature and the user habit feature, the initial recommended parameter set including at least one initial recommended parameter; Based on the application semantic features, the user habit features, the color block distribution sequence, and the user behavior sequence and application usage sequence within a preset time, determining the user demand features through a second determination model, wherein the user demand features are related information reflecting the text that the user may input in the current interface, and the second determination model is a machine learning model; Determining an estimated interaction frequency of each virtual key in the initial recommended parameter to be adaptively adjusted based on the user demand characteristics and the layout type of the sub-window in the initial recommended parameter to be adaptively adjusted; Adaptively adjusting the interaction size and reaction sensitivity of each virtual key based on the estimated interaction frequency to determine the recommended display parameter set; Determining the recommended display parameter set of the sub-window further includes: Constructing a third to-be-matched vector based on the user information, the foreground application, the application semantic feature, the input form feature, and the user habit feature; Based on the third vector to be matched, matching is performed in a second reference database to determine the recommended display parameter set; Determining a target display parameter in the recommended display parameter set based on a preset rule; A display instruction is generated based on the target display parameter to control a display module of the tablet computer to display the sub-window according to the target display parameter.
4. The tablet computer input management method according to claim 3, characterized in that: The method further comprises: Based on the color block distribution sequence and the foreground application, the application semantic feature is determined through a vector database.
Citation Information
Patent Citations
Intelligent digital assistants in multitasking environments
CN109783046B
Data processing method, terminal and storage medium
CN112463275A
Adjustment method and related device
CN116594550A
Deep learning code-free application layout optimization method and system based on user feedback
CN117193772A
Eye protection system and method for tablet computer
CN117472315A
