Module interaction method, system, device and storage medium based on tablet computer

Abstract

The embodiments of this specification provide a module interaction method, system, device and storage medium based on a tablet computer. The module interaction method based on a tablet computer includes: obtaining current scanning information based on a dictionary pen; determining display content based on the current scanning information; and controlling the display module to display the display content. The module interaction system based on a tablet computer includes a processor, a dictionary pen and a display module; wherein the processor is configured to execute the module interaction method based on a tablet computer. The method can also be executed after being read by a computer instruction stored in a computer-readable storage medium.

Description

Technical Field

[0001] The present invention relates to the field of information interaction, and in particular to a module interaction method, system, device and storage medium based on a tablet computer. Background Art

[0002] A dictionary pen is a portable electronic device with functions such as word search, translation, recording, note taking, and voice interaction. Users can scan and recognize text through the dictionary pen to obtain information such as the pronunciation, interpretation, and translation of the text. At present, the dictionary pen is limited by the size of the product, and the screen it carries is not large. Displaying various recognition contents on a small screen, such as displaying image recognition results, term explanations, voice and other information, has very limited operating space. In order to display the recognition content to the user, the existing dictionary pen adopts a method of communicating with an external screen to display the recognition content on a larger external screen for the user to view. However, the way in which the external screen displays the recognition content is often controlled by a preset program, and its display method is difficult to meet the different needs of a large number of users. For example, the method of displaying a large amount of content at the same time may make it difficult for users to grasp the key points. For another example, the method of displaying one by one may cause users to repeat cumbersome flipping operations, reducing the user's sense of use.

[0003] Therefore, it is desired to provide a module interaction method, system, device and storage medium based on a tablet computer to meet the dynamic reference needs of different users and improve the user's usage efficiency. Summary of the invention

[0004] In order to solve the problem of personalization and dynamism of the recognition content displayed by the existing dictionary pen, this specification provides a module interaction method, system, device and storage medium based on a tablet computer.

[0005] The present invention provides a module interaction method based on a tablet computer. The method is executed based on a tablet computer processor, and includes: obtaining current scanning information based on a dictionary pen; determining display content based on the current scanning information, the display content includes the current scanning information, and / or associative information corresponding to the current scanning information, the associative information is information associated with the current scanning information; and controlling a display module to display the display content.

[0006] The present invention provides a module interaction system based on a tablet computer, the system comprising: a processor, a dictionary pen and a display module. The processor is configured to: obtain current scanning information based on the dictionary pen; determine display content based on the current scanning information, the display content includes the current scanning information and / or associated information corresponding to the current scanning information, the associated information is information associated with the current scanning information; and control the display module to display the display content.

[0007] The present invention includes providing a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the module interaction method based on a tablet computer described in the above embodiment.

[0008] The beneficial effects brought by the above invention include but are not limited to: (1) Through the interaction between the dictionary pen and the tablet computer, the data scanned by the dictionary pen can be shared on the tablet computer, expanding the scope of use of the dictionary pen. (2) Determine the display content based on information such as user identity characteristics, so that the content and layout of the display content are more in line with the user's usage habits; (3) Determine the display parameters to display the display content, and intelligently adjust the image and text ratio and layout of the display content, and provide users with display content that users are interested in, so as to avoid the inconvenience of users sliding the screen to read due to too much display content, or too little display content that cannot meet the user's dynamically changing reference needs, thereby effectively improving the user's reading experience and usage efficiency, allowing users to better interact with the tablet computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is an exemplary flow chart of a module interaction method based on a tablet computer according to some embodiments of this specification;

[0010] Figure 2 is an exemplary schematic diagram of determining associative information according to some embodiments of this specification;

[0011] Figure 3 It is an exemplary schematic diagram of determining associative information based on intention probability distribution according to some embodiments of this specification. Implementation

[0012] The following is a brief introduction to the drawings required for describing the embodiments. The drawings do not represent all implementation methods.

[0013] The "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0014] Unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0015] Figure 1 is an exemplary flow chart of a module interaction method based on a tablet computer according to some embodiments of this specification. In some embodiments, process 100 is executed by a processor. Figure 1 As shown, process 100 includes the following steps.

[0016] Figure 1 is an exemplary flow chart of a module interaction method based on a tablet computer according to some embodiments of this specification. In some embodiments, process 100 is executed by a processor. Figure 1 As shown, process 100 includes the following steps.

[0017] Step 110, obtaining current scanning information based on the dictionary pen.

[0018] Scanning information refers to the information obtained by the dictionary pen through scanning. In some embodiments, the types of scanned information may include text, pictures, etc. The scanned information can be scanned or collected by the dictionary pen. For example, the dictionary pen can obtain text information through optical character recognition (OCR) scanning, and can also obtain picture information through photo scanning and recognition. It should be noted that the embodiments of this specification do not limit the language type of text information. For example, the text information can be Chinese text, or English text, etc. In addition, the embodiments of this specification do not limit the length of the text information. For example, the text information can be character text or sentence text.

[0019] In some embodiments, the processor can obtain the current scan information based on the dictionary pen in a variety of ways. For example, the dictionary pen can collect the current scan information through optical scanning recognition, and the dictionary pen can also collect the current scan information through voice recognition, key input, etc. After the dictionary pen completes the collection, the processor can establish a data connection with the dictionary pen in a wired or wireless manner to obtain the current scan information.

[0020] Step 120: Determine display content based on the current scanning information.

[0021] Display content refers to the content displayed by the display module. The form of display content includes but is not limited to text, symbols, images, and videos.

[0022] In some embodiments, the displayed content may include the current scanned information and / or the associated information corresponding to the current scanned information.

[0023] In some embodiments, the associated information is information associated with the current scanned information. For example, if the current scanned information scanned by the user is the idiom "Imitating Another without Success and Losing What Used to Be One's Own", the associated information may include: the interpretation, synonyms, antonyms, allusions, sources, etc. of this idiom.

[0024] In some embodiments, the associated information may include at least one type of association. The type of association refers to the direction of expanding and associating the scanned information. The type of association may include, but is not limited to, context association, interpretation association, synonym association, antonym association, allusion association, etc. Among them, context association can be used to expand the context content of the scanned information, interpretation association can be used to expand the interpretation content of the scanned information, and other types of associations are similar.

[0025] In some embodiments, the processor can determine the associated information of the current scanned information in various ways. For example, the processor can determine the associated information of the current scanned information based on the type of the current scanned information, based on a search engine or an offline database. When the type of the current scanned information is an idiom, the processor can query the offline dictionary database to determine the interpretation, synonyms, antonyms, allusions, sources, etc. of this idiom, and use this information as the associated information of the current scanned information. When the type of the current scanned information is a foreign language, the processor can query the offline foreign language database to determine the translation, synonyms, antonyms, part of speech, etc. of this foreign language, and use this information as the associated information of the current scanned information. Among them, the offline database can include the content of various dictionaries. For example, the offline database can include the content of various types of dictionaries such as "Modern Chinese Standard Dictionary", "Ancient Chinese Dictionary", "Modern Chinese Standard Dictionary", "Ancient Chinese Dictionary", etc.

[0026] In some embodiments, the processor can determine the user identity characteristics of the current user based on the current scanned information and the motion characteristics and pressure-sensitive characteristics of the dictionary pen, and further determine the associated information. For more content on this embodiment, please refer to Figure 2-3 and related descriptions.

[0027] In some embodiments, the processor can determine the display content in various ways. For example, the processor can directly use the current scanned information as the display content. For example, the processor can use the associated information of the current scanned information as the display content. For another example, the processor can use the current scanned information and its associated information as the display content. Among them, when the processor uses the current scanned information as the display content, the specific content of the current scanned content and the corresponding spelling, pronunciation, etc. can be displayed simultaneously. For example, when the current scanned information is the idiom "邯郸学步" (Literal: "Learning to Walk in Handan") and the display content is the current scanned information, the display content can include "邯郸学步" and "hán dān xué bù".

[0028] Step 130: Control the display module to display the display content.

[0029] In some embodiments, the display module can be a functional module in a tablet computer for displaying content. In some embodiments, the display module can include the display screen of the tablet computer. In some embodiments, the display module can also be a functional module in other external devices for displaying content, such as an e-reader, etc., which is not limited here.

[0030] The processor can control the display module to display the display content in various ways.

[0031] In some embodiments, the processor can control the display module to display the display content according to the target display mode.

[0032] In some embodiments, the processor can determine the target display mode of the display content according to the type of the display content. Among them, the type of the display content can include but is not limited to literal interpretation, word translation, sentence translation, etc. The display modes can include but are not limited to tiled display, list display, up and down sliding display (for example, view more content by sliding the page up and down), left and right sliding display (for example, view more content by sliding the page left and right), large font display, small font display, concise display (only display important content, for example, only display the interpretation of the scanned information, etc.), detailed display (for example, display the scanned information and all its associated information, etc.), key display (for example, perform key display by means of bold display, underlined display, color deepening display, etc.). In some embodiments, the corresponding relationship between different types of display content and different display modes can be preset based on historical data or prior knowledge. Correspondingly, the processor can determine the target display mode of the display content based on the type of the display content and this corresponding relationship.

[0033] In some embodiments, the processor can also determine the historical display mode of the historical display content that is the same or similar to the display content as the target display mode of the current display content.

[0034] In some embodiments, the processor may determine the target display mode of the displayed content based on the actual scanning situation of the user using the dictionary pen. For example, when the user uses the dictionary pen to perform a quick scan (for example, the user uses the dictionary pen to scan multiple times within one minute), the target display mode may be determined to be a simplified display (for example, a shorter content is displayed to facilitate the user to quickly view it). When the user uses the dictionary pen to perform a slow scan (for example, the user's single scan time is long), the target display mode may be determined to be a detailed display to display more complete content for the user to read carefully.

[0035] In some embodiments, the processor may determine the target display mode of the displayed content based on the actual operation of the user on the display screen. For example, when the user zooms the displayed content on the display screen, it means that the user is not used to the layout of the currently displayed content. At this time, the processor may adjust the layout of the currently displayed content (for example, adjust it to a small font display) to meet the needs of the user. For another example, when the user uses heavier pressure to scan part of the original recorded content, it means that the original recorded content may be the part that the user focuses on. The processor may focus on displaying the displayed content corresponding to the original recorded content. Among them, the original recorded content scanned by the user may include but is not limited to mathematical formulas, text, words, sentences, etc.

[0036] In some embodiments, the processor may control the display module to display the presentation content according to the target display parameters.

[0037] Display parameters refer to parameters related to displaying content by the display module. In some embodiments, the display parameters may include image-text ratio and typesetting parameters.

[0038] The image-text ratio refers to the display ratio of image information (such as pictures, videos, etc.) and text information in the displayed content.

[0039] The layout parameters refer to parameters related to the layout of the displayed content. In some embodiments, the layout parameters may include the font size, font type, font color, etc. of the displayed content.

[0040] The target display parameter refers to the display parameter for displaying the display content for the current user. The current user refers to the user who is currently using the dictionary pen.

[0041] In some embodiments, the processor may determine the target display parameter in a variety of ways. For example, the processor may determine a preset value as the target display parameter. Among them, the preset value may be determined based on prior experience. In some embodiments, the processor may perform statistical analysis based on the historical display data of the display module to obtain the total historical viewing time of different display parameters, and determine the display parameter with the longest total historical viewing time as the target display parameter. The historical display data may be the usage data of the display module in the previous few uses or in the recent period of time. The total historical viewing time refers to the total time of multiple readings of different display contents. The longer the total historical viewing time of the display parameter, the more the user tends to use the display parameter.

[0042] In some embodiments, the ratio of images to text may be related to the cognitive ability of the user. Accordingly, the processor may determine the ratio of images to text according to the cognitive ability of the current user.

[0043] User cognitive ability can represent the cognitive level of the current user. User cognitive ability can be expressed by a numerical value or a level. For example, the larger the numerical value, the higher the cognitive level of the current user. For more information about user cognitive ability, please refer to Figure 2 and its related description.

[0044] In some embodiments, the correspondence between different user cognitive abilities and different image-text ratios can be determined based on historical data or prior knowledge. For example, if the current user's cognitive ability is higher, the user may be more inclined to use an image-text ratio with less image information and more text information to reduce the frequency of page turning and obtain more text content; if the current user's cognitive ability is lower, the user may be more inclined to use an image-text ratio with less text information and more image information to quickly determine whether the displayed content is of interest.

[0045] In some embodiments, the processor may determine the image-text ratio of the content displayed for the current user based on the user cognitive ability of the current user and the aforementioned correspondence.

[0046] In some embodiments of the present specification, the image-text ratio is related to the user's cognitive ability, and the image-text ratio of the displayed content can be adaptively adjusted according to the user's cognitive ability to provide the user with display content with a more appropriate image-text ratio, thereby improving the user's usage experience.

[0047] In some embodiments, the layout parameters further include screen occupation spaces of association information of different association types.

[0048] Screen occupied space refers to the screen area occupied by associative information when it is displayed on the screen. In some embodiments, associative information of different associative types can be displayed simultaneously on the display screen. In some embodiments, the screen occupied space of associative information of different associative types can be different.

[0049] In some embodiments, the screen occupied space of associative information of different associative types may be related to the probability value of associative information of different associative types in the intention probability distribution, and the higher the probability value, the higher the screen occupied space. For example, the probability values ​​of the current user for contextual association, interpretation association, and synonym association in the intention probability distribution are p1, p2, and p3, respectively, then the screen occupied space of contextual association is positively correlated to the probability value p1, the screen occupied space of interpretation association is positively correlated to the probability value p2, and the screen occupied space of synonym association is positively correlated to the probability value p3. Accordingly, in some embodiments, the processor may determine the screen occupied space of associative information of different association types based on the probability value of associative information of different association types in the intention probability distribution.

[0050] For more information on the probability distribution of intention, see Figure 3 and its related description.

[0051] In some embodiments, the processor may determine the target association type, normalize the probability value corresponding to the target association type according to the intention probability distribution, and allocate the screen space occupied by the associative information of different target association types according to the normalized probability value. Among them, the target association type refers to the associative type of the associative information that needs to be displayed as the display content. For example, if three types of associative information of association types need to be displayed on the screen, the probability values ​​of the three types of associative information of association types can be normalized, and then the screen space can be allocated according to the normalized probability value. The embodiments of this specification do not specifically limit the normalization method, and operations familiar to those skilled in the art can be used.

[0052] In some embodiments of the present description, the layout parameters include the screen space occupied by associative information of different associative types in the displayed content, so that associative information of different associative types can have different screen spaces occupied on the screen, so that the user can see more content of interest on the display page.

[0053] In some embodiments of the present specification, the display content is displayed according to the target display parameters. When displaying the display content, the reading habits and preferences of the current user can be considered to adaptively adjust the displayed display content to improve the user experience.

[0054] In some embodiments of the present specification, based on the current scanning information, the display content is determined, and the image-text ratio and layout of the display content can be intelligently adjusted, and the display content that the user is interested in can be provided to the user, thereby avoiding the inconvenience of users sliding the screen to read due to too much display content, or too little display content that cannot meet the user's dynamically changing reading needs, thereby effectively improving the user's reading experience and allowing the user to better interact with the tablet computer.

[0055] Figure 2 It is an exemplary schematic diagram of determining associative information according to some embodiments of this specification.

[0056] In some embodiments, the dictionary pen may include a motion sensor as well as a pressure sensor.

[0057] The motion sensor can be used to obtain the motion characteristics of the dictionary pen. The pressure sensor can be used to obtain the pressure characteristics of the dictionary pen.

[0058] The motion feature can be used to characterize the movement of the dictionary pen by the user. In some embodiments, the motion feature can include the movement direction and movement speed of the dictionary pen. The movement direction can include left-right movement (left-right here refers to the left-right direction on the plane where the original recorded content is located), up-down movement (up-down here refers to the up-down direction on the plane where the original recorded content is located), vertical movement (vertical here refers to the direction perpendicular to the plane where the original recorded content is located), etc. The movement speed can be the average speed of a single scan of the dictionary pen.

[0059] The pressure-sensing feature can be used to characterize data related to the pressure applied by the user to the tablet computer using the dictionary pen. For example, the pressure-sensing feature may include the pressure applied by the user to touch the tablet computer using the dictionary pen, the duration of the pressure applied, and the like.

[0060] See also Figure 2 In some embodiments, the processor may determine the user identity characteristics 240 of the current user based on the current scanning information 210 and the motion characteristics 220 and pressure characteristics 230 of the dictionary pen; and determine the association information 250 based on the current scanning information 210 and the user identity characteristics 240.

[0061] User identity features are feature information related to the identity of the current user. In some embodiments, user identity features may include basic user information, user usage preferences, etc.

[0062] In some embodiments, the basic information of the user may include the user's age, the user's learning stage, and other information, such as the user's language, which is not limited in this embodiment of the specification.

[0063] In some embodiments, the user's learning stage can be represented by the user's school age, for example, third grade of elementary school.

[0064] In some embodiments, the user's age and the user's learning stage can be entered into the dictionary pen by the user in advance. For example, when entering identity features, the user can first enter information such as age and learning stage, and then use the dictionary pen in a normal posture, and use the dictionary pen to scan multiple times with different use movements, pressures, etc. The processor can obtain the movement features and pressure features of the dictionary pen, and associate them with the age and learning stage entered by the user, and finally store them in a storage device or the cloud.

[0065] In some embodiments, the user usage preference may include the percentage of times the user scans different original recorded contents, the percentage of time the user watches different displayed contents, etc.

[0066] In some embodiments, the processor may perform statistical analysis on the historical data of the user's use of the dictionary pen and / or the tablet computer to determine the user's usage preferences. The historical data may include historical scanning information and historical display content. In some embodiments, the processor may update the user's usage preferences during the user's subsequent use of the dictionary pen and / or the tablet computer.

[0067] In some embodiments, the processor may associate the user's usage preferences with the user's basic information so that when the user's identity is identified, the user's usage preferences can be obtained accordingly. For example, the processor may count the historical motion features and historical pressure features of the dictionary pen when determining the user's usage preferences. Furthermore, the processor may respectively determine motion features that are similar to the historical motion features and pressure features that are similar to the historical pressure features, and associate the user's basic information corresponding to the similar motion features and similar pressure features with the user's usage preferences.

[0068] In some embodiments of the present specification, after associating the motion characteristics and pressure characteristics of the dictionary pen with the user's basic information and user usage preferences, it is helpful to identify the user's identity based on the motion characteristics and pressure characteristics of the dictionary pen during the actual use of the dictionary pen, so that when the tablet computer is used by different users, there is no need to repeatedly log in or switch accounts.

[0069] In some embodiments, the processor may determine the user identity characteristics of the current user by querying a user library based on the current scanning information and the motion characteristics and pressure characteristics of the dictionary pen.

[0070] The user database refers to a database including reference users and their corresponding user identity features, wherein the reference users refer to all users who have used the current dictionary pen.

[0071] In some embodiments, the user library can be pre-set based on the historical data of all users who have used the current dictionary pen. In some embodiments, the processor can obtain the basic user information input by the user and the motion characteristics and pressure characteristics obtained by the dictionary pen when the user uses the dictionary pen for the first time, associate the basic user information with the motion characteristics and pressure characteristics, and store them in the user library. In some embodiments, the processor can obtain the user's usage preferences during the user's use, store the user's usage preferences in the user library, and correspond them to the user's basic information. Whenever a new user registers, the processor can repeat the above process and store the relevant information of the new user in the user library to update the user library.

[0072] In some embodiments, the processor can calculate at least one first similarity based on the current scanning information and the user usage preferences of at least one reference user in the user library; calculate at least one second similarity based on the motion characteristics and pressure characteristics of the dictionary pen and the motion characteristics and pressure characteristics of at least one reference user in the user library using the dictionary pen; calculate at least one comprehensive similarity based on at least one first similarity and at least one second similarity; and, using at least one comprehensive similarity, determine the user identity characteristics of the current user.

[0073] The first similarity refers to the similarity between the current scan information and the user usage preference corresponding to the reference user in the user library. There may be a first similarity corresponding to the user usage preference of each reference user in the user library.

[0074] In some embodiments, the processor may construct a feature vector corresponding to the current scan information, and construct a feature vector corresponding to the user's usage preference, and obtain the first similarity by calculating the vector distance. The vector distance is negatively correlated with the first similarity, that is, the larger the vector distance, the smaller the first similarity. In some embodiments, the vector distance may include but is not limited to cosine distance, Euclidean distance, Manhattan distance, Mahalanobis distance, or Minkowski distance.

[0075] The second similarity refers to the similarity between the motion features and pressure features of the dictionary pen and the motion features and pressure features corresponding to the reference users in the user library. The motion features and pressure features of the dictionary pen and the motion features and pressure features of each reference user in the user library when using the dictionary pen can correspond to a second similarity. The calculation method of the second similarity is similar to the calculation method of the first similarity, which will not be repeated here.

[0076] The comprehensive similarity can represent the similarity between the current user and the reference users in the user database.

[0077] In some embodiments, the processor may determine the comprehensive similarity by weighted summation based on the first similarity and the second similarity. In some embodiments, the weighted weight may be preset by the system or manually.

[0078] In some embodiments, when the comprehensive similarity is determined by weighted summing the first similarity and the second similarity, the weights of the first similarity and the second similarity corresponding to different reference users may be different.

[0079] The weight distribution refers to the proportional distribution of the weighted weight corresponding to the first similarity and the weighted weight corresponding to the second similarity.

[0080] In some embodiments, the weight distribution may be related to the number of times the user usage preference is updated. Accordingly, the processor may determine the weight distribution of the first similarity and the second similarity corresponding to different reference users based on the number of times the user usage preference is updated corresponding to different reference users.

[0081] The number of times the user usage preference is updated refers to the number of times the processor updates the user usage preferences of different users. The number of times the user usage preference is updated can be determined by querying the storage device.

[0082] It is understandable that the more times the user usage preferences are updated, the more accurate the user usage preferences are, and the more accurately the first similarity can be calculated. Correspondingly, in some embodiments, the more times the user usage preferences corresponding to the reference user are updated, the greater the weight ratio of the first similarity in the weight allocation corresponding to the reference user. In some embodiments, the number of times the user usage preferences corresponding to different reference users are updated is different, and accordingly, when calculating the comprehensive similarity between different reference users and the current user, the weight allocation of the first similarity and the second similarity of different reference users is different. When the total weight value is constant, the greater the weight ratio of the first similarity in the weight allocation, the smaller the weight ratio of the second similarity.

[0083] In some embodiments of the present specification, the weight distribution of the first similarity and the second similarity corresponding to different reference users is related to the number of times the user usage preference is updated. When calculating the comprehensive similarity between different reference users and the current user, the impact of the number of times the user usage preferences of different reference users are updated on the calculation of the first similarity can be considered to obtain a more accurate weight distribution, thereby more accurately calculating the comprehensive similarity.

[0084] In some embodiments, the user identity characteristics may also include user cognitive abilities.

[0085] User cognitive ability can represent the current user's cognitive level. For more information about user cognitive ability, see Figure 1 and its related description.

[0086] In some embodiments, the processor may determine the user's cognitive ability based on the user's basic information, a high-frequency search content set, and a cognitive ability assessment model.

[0087] High-frequency search content refers to content that current users frequently search for. A high-frequency search content set includes one or more high-frequency search content.

[0088] In some embodiments, the processor can determine a high-frequency search content set by counting the historical scanning information and historical search information of the current user. For example, the processor can determine the information content with the highest number of scans in the historical scanning information as the high-frequency search content, and determine the search content with the highest number of searches in the historical search information as the high-frequency search content. The historical scanning information refers to the scanning information obtained during the user's historical scanning process. The historical search information refers to the search information during the user's historical search process, and the search information is the content directly searched by the user using the tablet computer.

[0089] The cognitive ability assessment model refers to a model used to determine the cognitive ability of a user. In some embodiments, the cognitive ability assessment model may be a machine learning model. For example, the cognitive ability assessment model may include any one or combination of a neural network (NN) model or other custom model structures.

[0090] In some embodiments, the input of the cognitive ability assessment model may include basic user information and a high-frequency search content set of the current user, and the output may include the user cognitive ability of the current user.

[0091] In some embodiments, the processor can perform training based on a large number of first training samples with first labels through various feasible training methods to obtain a trained cognitive ability assessment model. For example, training can be performed based on a gradient descent method. An exemplary training process includes: inputting a plurality of first training samples with first labels into an initial cognitive ability assessment model, constructing a loss function through the first label and the prediction result of the initial cognitive ability assessment model, updating the initial cognitive ability assessment model based on the iteration of the loss function, and completing the cognitive ability assessment model training when the loss function of the initial cognitive ability assessment model meets a preset iteration condition, wherein the preset iteration condition can be the convergence of the loss function, the number of iterations reaching a set value, etc.

[0092] In some embodiments, the first training sample may be the sample user's basic information and the sample high-frequency search content set of the sample user, and the first label of the first training sample may be the user cognitive ability of the sample user. In some embodiments, the first training sample may be obtained based on historical data, and the first label may be determined based on the actual cognitive ability annotation of the sample user. The actual cognitive ability may be obtained by conducting a cognitive assessment test on the sample user, and the cognitive assessment test may include any cognitive assessment test recognized by the public.

[0093] In some embodiments of this specification, the user's basic information and high-frequency search content set are processed through the cognitive ability evaluation model, and the self-learning ability of the machine learning model can be used to find patterns from a large amount of data, obtain the correlation between the user's basic information, high-frequency search content set and the user's cognitive ability, and improve the accuracy and efficiency of determining the user's cognitive ability. At the same time, according to the movement characteristics and pressure characteristics of the dictionary pen, the identity of the user can be identified, so that when the tablet is used by different users, there is no need to repeatedly log in or switch accounts.

[0094] In some embodiments, the processor may determine the associative information in a variety of ways based on the current scan information and the user's identity characteristics.

[0095] In some embodiments, the processor may input the current scanning information into a preset search engine to obtain one or more associated search data; filter the one or more associated search data to determine the filtered data, and determine the associative information based on the filtered data.

[0096] In some embodiments, the preset search engine may include an Internet search engine, such as a Google search engine, and may also include an offline database built into the tablet computer.

[0097] The associated search data refers to the search data associated with the current scan information. For example, when the processor searches through an Internet search engine, it can determine the content of each searched web page as the associated search data.

[0098] The filtered data refers to the associated search data remaining after the associated search data is filtered and excluded.

[0099] In some embodiments, the screening criteria for screening one or more associated search data may be determined based on the user identity characteristics. For example, the processor may classify the associated search data according to age and learning stage, and based on the classification results, filter out the associated search data that does not match the user identity characteristics.

[0100] In some embodiments, the processor may determine the screening data as the associative information. In some embodiments, the processor may determine part of the associated search data in the screening data as the associative information.

[0101] In some embodiments, the processor may determine whether the current scan information satisfies a search trigger condition, and in response to the current scan information satisfying the search trigger condition, input the current scan information into an Internet search engine to obtain associated search data.

[0102] The search trigger condition can be used to determine whether to input the current scan information into the Internet search engine. The search trigger condition can be preset, for example, the search trigger condition can be that no information associated with the current scan information is found in the offline database.

[0103] In some embodiments, the processor may determine the intention probability distribution of the current user based on the current scanning information, the user identity characteristics, and the historical scanning information sequence of the current user, and further determine the association information. For more information about this embodiment, see Figure 3 and its related description.

[0104] In some embodiments of the present specification, based on the current scanning information and user identity characteristics, associative information is determined, and when different users use the dictionary pen, the identity of the user can be automatically and quickly determined, and customized content can be provided to the user based on the user's identity to meet the user's usage needs.

[0105] Figure 3 It is an exemplary schematic diagram of determining associative information based on intention probability distribution according to some embodiments of this specification.

[0106] In some embodiments, the processor may determine the intention probability distribution of the current user based on the current scanning information, the user identity characteristics and the historical scanning information sequence of the current user; and determine the association information based on the current scanning information, the user identity characteristics and the intention probability distribution.

[0107] The historical scanning information sequence is a sequence composed of multiple historical scanning information of the current user performing multiple scans using the dictionary pen. In some embodiments, the historical scanning information sequence may include multiple historical scanning information obtained by the current user during the process of performing multiple scans when the dictionary pen is activated.

[0108] In some embodiments, the processor may obtain a historical scanning information sequence by summarizing it in a local storage device or a cloud storage device of the dictionary pen.

[0109] The intention probability distribution refers to the probability distribution of the current user viewing different types of associative content (i.e. associative information). The intention probability can indicate the current user's viewing tendency for different associative information. The greater the intention probability, the higher the viewing tendency. For more information about associative information, see Figure 1 and its related description.

[0110] In some embodiments, the processor may determine the intention probability distribution in a variety of ways. For example, the processor may determine at least one associative information corresponding to the current user by querying a first preset table based on the current scanning information and historical scanning information sequence of the current user, and then determine the intention probability corresponding to the at least one associative information by querying a second preset table based on the user identity feature and the at least one associative information, to construct the intention probability distribution.

[0111] In some embodiments, the first preset table may include historical scanning information sequences, current scanning information, and corresponding multiple association information of different users. In some embodiments, the second preset table may include different user identity features, association information, and corresponding intention probabilities. In some embodiments, the first preset table and the second preset table may be preset based on historical data or prior knowledge.

[0112] See also Figure 3 In some embodiments, the processor may determine the intention probability distribution 340 through the intention probability determination model 320 based on the current scanning information 210 , the user identity feature 240 , and the historical scanning information sequence 311 .

[0113] The intention probability determination model refers to a model for determining the intention probability distribution. In some embodiments, the intention probability determination model can be a machine learning model. For example, the intention probability determination model can include any one or combination of NN models or other customized model structures.

[0114] See also Figure 3 In some embodiments, the intention probability determination model 320 may include a semantic type recognition layer 321 and a probability determination layer 322 .

[0115] The semantic type recognition layer can be used to determine the semantic type of the current scan information. Figure 3 In some embodiments, the input of the semantic type recognition layer may include the current scan information 210, and the output may include the semantic type 330 of the current scan information.

[0116] The semantic type refers to the distinction of the semantics of the current scan information. In some embodiments, the semantic type can be divided by subject, for example, the semantic type can include subject types such as Chinese, mathematics, and English. In some embodiments, the semantic type divided by subject can be further subdivided on the basis of the subject. For example, if the semantic type of the current scan information is the subject of Chinese, then the current scan information can be further subdivided into idioms, classical Chinese, modern poems, ancient poems, and other types.

[0117] The probability determination layer can be used to determine the probability distribution of intentions. Figure 3In some embodiments, the input of the probability determination layer may include the current scanning information 210 , the semantic type 330 of the current scanning information 210 , the user identity feature 240 , and the historical scanning information sequence 311 , and the output may include the intention probability distribution 340 .

[0118] See also Figure 3 In some embodiments, the input of the probability determination layer 322 may also include the historical scanning trajectory 312 of the current user.

[0119] The historical scanning trajectory refers to the motion trajectory formed by multiple scans in the process of the current user obtaining the current scanning information. For example, when the current scanning information is distributed at intervals in the information carrier, the historical scanning trajectory may be moving from left to right on the plane where the original recorded content is located and scanning x centimeters, pausing for a period of time, continuing to move from left to right and scanning x centimeters, and pausing for a period of time again. The information carrier may include books, textbooks, etc.

[0120] In some embodiments, the process of the dictionary pen approaching the original recorded content and scanning it by pressing a scanning control element (eg, a key, a switch, etc.) can be considered as one scan. When the dictionary pen moves away from the original recorded content or the scanning control element is turned off, the scan ends.

[0121] In some embodiments, the historical scanning trajectory can represent the position distribution of the current scanning information on the information carrier. For example, when the historical scanning trajectory is scanning a certain distance, pausing for a period of time, and then scanning again, it can be determined that the current scanning information is distributed at intervals on the information carrier. For another example, when the historical scanning trajectory is continuously scanning a certain distance, it can be determined that the current scanning information is continuously distributed on the information carrier.

[0122] In some embodiments, the historical scanning trajectory can characterize the user's page turning operation. It is understandable that when the scanning information corresponding to the same spatial position of the dictionary pen is different, it can be said that the current user has performed a page turning operation. Correspondingly, the processor can confirm the number of page turning operations of the current user through the historical scanning trajectory. For example, the spatial position of the fourth scanning trajectory is the same as that of the fifth scanning trajectory, but the scanning information corresponding to the two scanning trajectories is different, indicating that the current user has performed a page turning. In some embodiments, when other conditions (for example, user identity characteristics, information carriers, etc.) are the same, when the current user has a large number of page turnings, it indicates that the current user may want some simple interpretation associations, so the page turning is faster. At this time, the intention probability of the interpretation association in the intention probability distribution can be increased. The current user does not have a large number of page turnings, which indicates that the current user may want more comprehensive associative content, so the page turning is slower. At this time, the intention probability of context association, synonym association, and antonym association in the intention probability distribution can be increased.

[0123] In some embodiments, the processor can obtain the historical scanning trajectory by analyzing the motion features when the dictionary pen obtains the current scanning information. For example, the motion features when the dictionary pen obtains the current scanning information can be in a sequence form, and different moments can correspond to a motion feature. The processor can obtain the historical motion trajectory of the dictionary pen by analyzing the moving direction and moving speed of adjacent motion features.

[0124] In some embodiments of the present specification, the input of the probability determination layer also includes the historical scanning trajectory of the current user, so that the intention probability distribution can be determined more accurately.

[0125] In some embodiments, the semantic type recognition layer and the probability determination layer in the intention probability determination model can be obtained through separate training.

[0126] In some embodiments, the processor may train a semantic type recognition layer based on a large number of second training samples with second labels in a variety of ways, for example, training may be performed based on a gradient descent method.

[0127] In some embodiments, the second training sample may be sample scan information, and the second label of the second training sample may be a semantic type corresponding to the sample scan information. In some embodiments, the second training sample may be obtained based on historical data, and the second label may be determined by manual annotation.

[0128] In some embodiments, the processor may train a probability determination layer based on a large number of third training samples with third labels in a variety of ways, for example, training may be performed based on a gradient descent method.

[0129] In some embodiments, the third training sample may be sample scanning information, a semantic type corresponding to the sample scanning information, a user identity feature of a sample user, a historical scanning information sequence of the sample user, and a historical scanning trajectory of the sample user, and the third label of the third training sample may be an intention probability distribution. In some embodiments, the third training sample may be obtained based on historical data.

[0130] In some embodiments, the third label can be determined by manual annotation. For example, it can be assumed that at the historical moment t0 corresponding to the sample scanning information of a third training sample, two kinds of associative information, context association and interpretation association, are displayed to the user, and then the third label can be annotated for the third training sample according to the time interval between the historical moment t0 and the moment t1 when the user actively updates the display screen. Exemplary annotation methods include: the larger the time interval between moment t1 and moment t0, the closer the label value corresponding to the context association and interpretation association in the intention probability distribution is to 1, and the closer the label value corresponding to other associations is to 0, and the label value is the probability of the associative content in the intention probability distribution. Another exemplary annotation method includes: in response to the time interval between moment t1 and moment t0 being greater than the time threshold, the label values ​​corresponding to the context association and interpretation association in the intention probability distribution are respectively annotated as 1, and the label values ​​corresponding to other associative content are 0; conversely, the label values ​​corresponding to the context association and interpretation association in the intention probability distribution are respectively annotated as 0, and the label values ​​corresponding to other associative content are 1.

[0131] In some embodiments, the training process of the semantic type recognition layer and the probability determination layer in the intention probability determination model is similar to the training process of the cognitive ability assessment model, and its implementation method can refer to the implementation method of the training process of the cognitive ability assessment model.

[0132] In some embodiments of the present specification, the current scanning information, user identity characteristics, and historical scanning information sequences are processed through an intention probability determination model. The self-learning ability of the machine learning model can be used to find patterns from a large amount of data, obtain the correlation between the current scanning information, user identity characteristics, historical scanning information sequences and the intention probability distribution, and improve the accuracy and efficiency of determining the intention probability distribution.

[0133] See also Figure 3 In some embodiments, the processor may determine the association information 250 based on the current scanning information 210 , the user identity feature 240 , and the intention probability distribution 340 .

[0134] In some embodiments, the processor may input the current scanning information into a preset search engine to obtain one or more associated search data; perform a first screening on the one or more associated search data based on the user identity characteristics to determine the first screening data; perform a second screening on the first screening data based on the intention probability distribution to determine the second screening data, and determine the association information based on the second screening data. The process of performing the first screening to obtain the first screening data is similar to Figure 2 The screening process is the same as described above, see Figure 2 .

[0135] The first screening data refers to one or more key search data obtained after the first screening. The second screening data refers to one or more key search data obtained after the first screening and the second screening.

[0136] In some embodiments, the processor may classify the first screening data according to the association type, retain the first screening data whose intention probability in the intention probability distribution is greater than the probability threshold, and obtain the second screening data. The probability threshold may be preset.

[0137] In some embodiments, the processor may determine the second screening data as associative information. In some embodiments, the processor may determine part of the associated search data in the second screening data as associative information.

[0138] In some embodiments of the present specification, by determining association information based on current scanning information, user identity characteristics, and intention probability distribution, association information that better meets user needs can be obtained, which is conducive to determining appropriate display content.

[0139] The present specification also provides a module interaction device based on a tablet computer, which includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; and the at least one processor is used to execute at least part of the computer instructions to implement the module interaction method based on the tablet computer described in any one of the above embodiments.

[0140] The present specification also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the module interaction method based on a tablet computer as described in any one of the above embodiments.

[0141] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0142] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0143] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0144] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0145] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining the digits. Although the numerical domains and parameters used to confirm the breadth of the range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0146] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification is hereby incorporated by reference in its entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0147] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A module interaction method based on a tablet computer, the method being executed by a processor, characterized in that: The method comprises: Get current scan information based on dictionary pen; Based on the current scanning information, determine display content, the display content includes the current scanning information, and / or associated information corresponding to the current scanning information, the associated information is information associated with the current scanning information; Controlling the display module to display the display content; The method for determining the association information includes: Determining a user identity feature of a current user based on the current scanning information and the motion features and pressure features of the dictionary pen; Based on the current scanning information, the user identity characteristics and the historical scanning information sequence of the current user, the intention probability distribution of the current user is determined through an intention probability determination model; the intention probability determination model includes a semantic type recognition layer and a probability determination layer; the semantic type recognition layer determines the semantic type of the current scanning information based on the current scanning information; the semantic type refers to the distinction of the semantics of the current scanning information; the probability determination layer determines the intention probability distribution based on the current scanning information, the semantic type of the current scanning information, the user identity characteristics of the current user, the historical scanning information sequence, and the historical scanning trajectory; the historical scanning trajectory refers to the motion trajectory formed by multiple scans in the process of the current user obtaining the current scanning information; the intention probability distribution refers to the probability distribution of the current user viewing content of different association types; The association information is determined based on the current scanning information, the user identity feature, and the intention probability distribution.

2. The method according to claim 1, characterized in that The controlling the display module to display the display content includes: The display module is controlled to display the display content according to the target display parameters.

3. A tablet computer-based module interaction system, characterized in that: The invention comprises a processor, a dictionary pen and a display module, wherein the processor is configured to: Acquiring current scanning information based on the dictionary pen; Determine display content based on the current scanning information, the display content includes the current scanning information and / or associated information corresponding to the current scanning information, the associated information is information associated with the current scanning information; Controlling the display module to display the display content; The dictionary pen includes a motion sensor and a pressure sensor, and the processor is further configured to: Determining a user identity feature of a current user based on the current scanning information and the motion features and pressure features of the dictionary pen; Based on the current scanning information, the user identity characteristics and the historical scanning information sequence of the current user, the intention probability distribution of the current user is determined through an intention probability determination model; the intention probability determination model includes a semantic type recognition layer and a probability determination layer; the semantic type recognition layer determines the semantic type of the current scanning information based on the current scanning information; the semantic type refers to the distinction of the semantics of the current scanning information; the probability determination layer determines the intention probability distribution based on the current scanning information, the semantic type of the current scanning information, the user identity characteristics of the current user, the historical scanning information sequence, and the historical scanning trajectory; the historical scanning trajectory refers to the motion trajectory formed by multiple scans in the process of the current user obtaining the current scanning information; the intention probability distribution refers to the probability distribution of the current user viewing content of different association types; The association information is determined based on the current scanning information, the user identity feature, and the intention probability distribution.

4. The system according to claim 3, characterized in that The processor is further configured to: The display module is controlled to display the display content according to the target display parameters.

5. A module interaction device based on a tablet computer, comprising a processor, wherein the processor is used to execute the module interaction method based on a tablet computer according to any one of claims 1 to 2.

6. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the module interaction method based on a tablet computer as described in any one of claims 1 to 2.

Images