Online reading multi-mode auxiliary bookmark marking method and system for visually impaired people

Abstract

The invention relates to the field of auxiliary reading, in particular to a multi-mode auxiliary bookmark marking method and system for online reading of visually impaired people. The method comprises the following steps: acquiring a pause signal, and determining text information corresponding to a pause moment based on the pause signal; analyzing the reading text according to the text information, and determining a marking range; performing sentence definition on the content in the marking range to obtain the sentence of the single-sentence text in the marking range, and determining the possibility of text marking according to the sentence; a number of tag types may be designed for a user to select according to the text tag. The pause signal of the visually impaired people is captured, the operation is immediately responded, the inconvenience of the user in response waiting is avoided, and the experience feeling is improved. The reading position at the pause moment is accurately recorded, and the accuracy of bookmark marking is ensured. By analyzing the reading text, the interested content is automatically recognized, the trouble of manual searching and marking is reduced, and the efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of assisted reading, and in particular to a bookmarking method and system for multimodal assisted online reading for the visually impaired. Background Art

[0002] With the continuous advancement of science and technology, the reading needs of the visually impaired are receiving more and more attention. From traditional paper books to e-books, and then to modern multimodal reading devices, the visually impaired have more choices in enjoying information acquisition and entertainment content. Multimodal technology improves the reading experience of the visually impaired by combining auxiliary tools of different senses such as voice, touch, and vision. In a multimodal assisted reading environment, bookmarks and marking functions are undoubtedly the key to improving user experience. Especially for long text content, being able to help users mark specific reading positions, conduct content review, sentiment analysis, or content relevance analysis are all core technical requirements to help the visually impaired read effectively.

[0003] In the prior art, the bookmark or marking function usually relies on a fixed marking position, but these methods lack sufficient personalization and intelligence for visually impaired users. Especially when users are required to accurately mark a certain part of the content, it often takes a lot of time to understand and assist in the implementation, which greatly affects the reading experience of the visually impaired. Summary of the invention

[0004] The present application provides a multi-modal assisted bookmarking method and system for online reading for the visually impaired to solve the above-mentioned problems.

[0005] In a first aspect, the present application provides a multimodal assisted bookmarking method for online reading for visually impaired people, the method comprising: Obtaining a pause signal, and determining text information corresponding to the pause moment based on the pause signal; Analyze and read the text according to the text information to determine the marking range; Defining the sentence type of the content within the marked range, obtaining the sentence type of a single sentence within the marked range, and determining the possibility of text marking according to the sentence type; Several label types may be designed based on the text mark for the user to choose from.

[0006] Through this solution, by capturing the pause signal of the visually impaired, the operation is responded to immediately, avoiding the inconvenience of waiting for the response and improving the user experience. The reading position at the moment of pause is accurately recorded to ensure the accuracy of bookmarking. By analyzing the reading text, the content of interest is automatically identified, reducing the trouble of manual search and marking and improving efficiency. Sentence definition helps to analyze the semantics and structure of the text, making bookmarking more intelligent and meaningful, and easier to review and identify the marked content. Determining the possibility of text tagging based on sentence type makes the provided tags closer to reading needs and improves the practicality and relevance of tags. Provide a variety of tag types for users to choose from, and choose appropriate tags according to their needs and preferences to increase user autonomy and personalized experience.

[0007] Optionally, analyzing and reading the text to determine the marking range according to the text information includes: Get the reading speed before the pause moment; Determining the text reading length per unit time according to the reading speed; Analyze the text reading length and the text information, analyze the reading text, and determine the marking range.

[0008] Through this solution, by obtaining the reading speed, the reading habits and rhythm can be identified, so as to better predict the text content that has been read before the pause moment. Calculating the text reading length per unit time helps to determine a rough text range, which is the content read before the pause moment. By analyzing the text reading length and text information, the content of interest can be more accurately identified, providing a basis for determining the marking range. Analyzing the reading text helps to identify the semantics and structure of the text, so as to more accurately determine the part of the text that is of interest. Based on the analysis results, a suitable marking range is determined, which can be the full text, partial text or single sentence text, which helps to bookmark more effectively.

[0009] Optionally, analyzing the text reading length and the text information, analyzing the reading text, and determining the marking range includes: Acquire device movement data, analyze the device movement data, and determine the device pick-up time; Determining the operation time of the user according to the device picking time and the pause time; Determining the text reading record within the operation time according to the text reading length; A marking range is determined according to the text information and the text reading record.

[0010] This solution obtains device movement data and identifies the user's behavior before picking up the device, which helps determine the time when the user starts reading. By analyzing the device movement data, the device pick-up time can be more accurately determined, thereby more accurately calculating the reading time. Combining the device pick-up time and the pause time, the operation time from picking up the device to pausing the operation is calculated. The text content read during this period is determined by the reading speed and operation time. Combining the user's reading record and text information, the content of interest to the user can be more accurately identified, thereby determining a suitable marking range.

[0011] Optionally, determining the text tag according to the sentence type may include: For each single sentence text, determining the text sentiment of the single sentence text according to the sentence type; Analyzing the text reading record and the text sentiment to determine the sentiment connection strength between the single sentence text and the context; Determining the content connection strength between the single sentence text and the context according to the sentence type; The possibility of text tagging is determined according to the emotional connection strength and the content connection strength.

[0012] Through this scheme, through sentence definition, the basic functions of each sentence such as statement, question or expression of emotion are identified, which is helpful for sentiment analysis and judgment of content connection strength. Text sentiment analysis can reveal the emotional tendency of a single sentence text, such as positive, negative or neutral, which helps to identify the emotional response to the text. By analyzing the emotional connection strength between a single sentence text and the context, the emotional turning point or consistency in the text is identified, so as to better analyze the emotional changes in the reading process. The analysis of content connection strength helps to judge the degree of relevance between a single sentence text and the context in terms of topic and content, and helps to determine the coherence of the text and the user's interest points. Comprehensive analysis can provide a comprehensive perspective, which helps to evaluate the closeness of a single sentence text to the context in terms of emotion and content, so as to more accurately predict the tagging intention. The final determination of text tagging may ensure that the bookmark tagging suggestions match the actual reading interests and emotional responses, thereby improving the accuracy of tagging and user satisfaction.

[0013] Optionally, determining the text tag possibility according to the emotional connection strength and the content connection strength includes: According to the content connection strength, determining whether there are at least two single-sentence texts that have inseparable sentence meaning connections; If so, it is possible to mark the at least two single-sentence texts that are inseparable from each other as a text; If not, determining the emotional coherence between the single sentence text and the rest of the text according to the emotional connection strength; Based on the sentiment coherence, a text tagging possibility is determined.

[0014] Through this scheme, by analyzing the content connection strength between at least two single-sentence texts, the coherence and logical relationship in the text are identified, which helps to identify points of interest and information needs during the reading process. If there is an inseparable sentence connection, these texts are considered as a whole for marking, which helps to better preserve and identify related information content. Marking texts with inseparable sentence connections as bookmarks helps to quickly review and access important information and improve reading efficiency. Analyzing the emotional coherence between a single sentence text and the rest of the text helps to identify emotional changes and reactions during the reading process, thereby providing a more personalized reading experience. By determining emotional coherence, identifying emotional turning points or consistency in the text helps to better analyze the text content and emotional background. Based on the results of the emotional coherence analysis, the marking intention is more accurately predicted, thereby providing more precise and personalized bookmarking suggestions.

[0015] Optionally, determining the text tag according to the sentence type may include: Analyzing the reading text to determine text attributes; Determining the content type within the marked range according to the text attribute; Analyzing the pause signal according to the content type to determine whether the pause signal is a bookmark establishment signal; If so, the text tagging possibility is determined based on the sentence type.

[0016] Through this solution, the structural and content characteristics of the text are obtained by analyzing the reading text. According to the results of the text attribute analysis, the content types within the marked range, such as descriptive content, illustrative content, opinion content, etc., are identified, which helps to identify the information categories that the user is interested in. By analyzing the pause signal, it is determined whether the intention is to create a bookmark. It helps to provide more accurate marking suggestions to meet the needs of the reading process. If it is determined that the pause signal is a bookmark establishment signal, the text is further analyzed according to the sentence type to provide the user with more specific marking suggestions. It helps to quickly locate and review important information content.

[0017] Optionally, determining the marking range according to the text information and the text reading record includes: Reading records based on the text to determine the integrity of the text; Comparing the text integrity with a preset judgment threshold, and if the text integrity is lower than the preset judgment threshold, analyzing the text reading record to determine whether the sentence structure is correct; If it is accurate, then based on the punctuation, a supplementary text is obtained; A marking range is determined according to the text information, the supplementary text and the text reading record.

[0018] This solution analyzes the text reading record to determine whether all parts of the text have been read, thereby ensuring a holistic analysis of the text. Comparing the text completeness index with the preset judgment threshold helps to quickly identify whether the user's reading has reached the expected completeness, thereby determining whether further analysis is needed. Analyzing sentence accuracy helps to identify whether the reading pattern matches the syntactic structure of the text, and helps to improve the recognition of reading intent. Through punctuation analysis, unread or omitted parts of the text are inferred, thereby supplementing the reading record and improving the completeness of the record. Combining text information, supplementary text, and reading records to more accurately determine the range of text you want to mark helps to quickly locate and review content of interest.

[0019] Optionally, determining the marking range according to the text information, the supplementary text and the text reading record includes: Obtain the user's historical reading records; Analyze the historical reading records to determine historical marks and historical pause signals; Analyze the historical pause signal and the corresponding historical mark to determine the historical picking time of the user; Acquire historical reading settings; determine the historical reading speed corresponding to each historical pause signal according to the historical reading settings; Determine a historical reading record of the historical operation duration corresponding to each historical pause signal according to the historical reading speed and the historical fetching time; Determining the reading reaction time of the user according to the historical reading records and the corresponding historical marks; Determine the reaction reading record within the reading reaction time according to the text reading length; A marking range is determined according to the text information, the supplementary text, the reaction reading record and the text reading record.

[0020] Through this solution, historical reading data is collected to provide basic information for analyzing reading habits and preferences. By analyzing historical data, identifying marking behaviors and pause points during reading helps analyze reading patterns and points of interest. Determining the specific time points of pauses in historical reading helps calculate reading speed and reaction time. Identifying personalized settings in historical reading, such as reading speed, helps analyze reading behavior more accurately. Calculating the reading speed at each pause helps infer the degree of concentration when reading text. Determining the reading duration at each pause helps identify behavioral patterns during reading. Determining the reaction time to text content through historical data analysis helps identify the thought process when reading. Determining the text content read within the reaction time helps identify changes in interest during reading. Combining text content, supplementary text, and reading behavior to determine the range of text you want to mark helps improve the accuracy and satisfaction of bookmarking.

[0021] Optionally, determining the content connection strength between the single sentence text and the context according to the sentence type includes: The sentiment relevance between the single sentence text and the context is quantified through the sentiment word intensity matrix and the context attention weight.

[0022] Through dependency syntactic tree and latent semantic analysis, the logical correlation between each adjacent sentence text is modeled.

[0023] According to the sentiment relevance and the inter-sentence logical relevance, a joint matrix is ​​established to determine the content connection strength between the single sentence text and the context.

[0024] Through this solution, the structure and content of the text can be more accurately identified through word segmentation and part-of-speech tagging. The sentiment word strength matrix helps to identify and quantify the sentiment tendency in the text, and the context attention weight ensures that the results of sentiment analysis are closer to actual feelings and reading interests. Dependency syntactic tree analysis provides grammatical structure information within the sentence, and latent semantic analysis reveals the semantic relationship between sentences. The two together provide a basis for analyzing the logical correlation between sentences. The joint matrix combines sentiment correlation and inter-sentence logical correlation to provide a comprehensive evaluation index for the content connection between a single sentence text and the context. By analyzing the joint matrix, the content connection strength between a single sentence text and the context is determined, and accurate bookmarking suggestions are provided, which helps users manage and collect reading content more effectively.

[0025] In a second aspect, the present application provides a bookmarking system for multimodal online reading assistance for visually impaired people, the system comprising: An information determination module, used for obtaining a pause signal, and determining text information corresponding to the pause moment based on the pause signal; A range determination module, used to analyze and read the text and determine the marking range according to the text information; A possibility analysis module is used to define the sentence type of the content within the marked range, obtain the sentence type of a single sentence text within the marked range, and determine the possibility of text marking according to the sentence type; The label design module is used to design several label types according to the text mark for users to choose.

[0026] Optionally, when the range determination module analyzes and reads the text according to the text information to determine the marking range, it is used to: Get the reading speed before the pause moment; Determining the text reading length per unit time according to the reading speed; Analyze the text reading length and the text information, analyze the reading text, and determine the marking range.

[0027] Optionally, the range determination module analyzes the text reading length and the text information, analyzes the reading text, and determines the marking range, and is used to: Acquire device movement data, analyze the device movement data, and determine the device pick-up time; Determining the operation time of the user according to the device picking time and the pause time; Determining the text reading record within the operation time according to the text reading length; A marking range is determined according to the text information and the text reading record.

[0028] Optionally, when the possibility analysis module determines that the text tag is possible according to the sentence type, it is used to: For each single sentence text, determining the text sentiment of the single sentence text according to the sentence type; Analyzing the text reading record and the text sentiment to determine the sentiment connection strength between the single sentence text and the context; Determining the content connection strength between the single sentence text and the context according to the sentence type; The possibility of text tagging is determined according to the emotional connection strength and the content connection strength.

[0029] Optionally, when the possibility analysis module determines that the text tag is possible according to the sentiment connection strength and the content connection strength, it is used to: According to the content connection strength, determining whether there are at least two single-sentence texts that have inseparable sentence meaning connections; If so, it is possible to mark the at least two single-sentence texts that are inseparable from each other as a text; If not, determining the emotional coherence between the single sentence text and the rest of the text according to the emotional connection strength; Based on the sentiment coherence, a text tagging possibility is determined.

[0030] Optionally, when the possibility analysis module determines that the text tag is possible according to the sentence type, it is used to: Analyzing the reading text to determine text attributes; Determining the content type within the marked range according to the text attribute; Analyzing the pause signal according to the content type to determine whether the pause signal is a bookmark establishment signal; If so, the text tagging possibility is determined based on the sentence type.

[0031] Optionally, when the range determination module determines the marking range according to the text information and the text reading record, it is used to: Reading records based on the text to determine the integrity of the text; Comparing the text integrity with a preset judgment threshold, and if the text integrity is lower than the preset judgment threshold, analyzing the text reading record to determine whether the sentence structure is correct; If it is accurate, then based on the punctuation, a supplementary text is obtained; A marking range is determined according to the text information, the supplementary text and the text reading record.

[0032] Optionally, when the range determination module determines the marking range according to the text information, the supplementary text and the text reading record, it is used to: Obtain the user's historical reading records; Analyze the historical reading records to determine historical marks and historical pause signals; Analyze the historical pause signal and the corresponding historical mark to determine the historical picking time of the user; Acquire historical reading settings; determine the historical reading speed corresponding to each historical pause signal according to the historical reading settings; Determine a historical reading record of the historical operation duration corresponding to each historical pause signal according to the historical reading speed and the historical fetching time; Determining the reading reaction time of the user according to the historical reading records and the corresponding historical marks; Determine the reaction reading record within the reading reaction time according to the text reading length; A marking range is determined according to the text information, the supplementary text, the reaction reading record and the text reading record.

[0033] Optionally, when the possible analysis module determines the content connection strength between the single sentence text and the context according to the sentence type, it is used to: The sentiment relevance between the single sentence text and the context is quantified through the sentiment word intensity matrix and the context attention weight.

[0034] Through dependency syntactic tree and latent semantic analysis, the logical correlation between each adjacent sentence text is modeled.

[0035] According to the sentiment relevance and the inter-sentence logical relevance, a joint matrix is ​​established to determine the content connection strength between the single sentence text and the context. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application; Figure 2 A flowchart of a multi-modal assisted bookmarking method for online reading for the visually impaired provided in one embodiment of the present application; Figure 3 A schematic diagram of the structure of a bookmarking system for multimodal online reading assistance for the visually impaired provided in one embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0040] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0041] The present application provides a multimodal assisted bookmarking method and system for online reading for the visually impaired, which obtains a pause signal, determines the text information corresponding to the pause time based on the pause signal; analyzes the reading text according to the text information and determines the marking range; performs sentence definition on the content within the marking range, obtains the sentence nature of the single sentence text within the marking range, and determines the text marking possibility according to the sentence nature; designs several tag types for users to choose according to the text marking possibility. By capturing the pause signal of the visually impaired, the operation is responded to immediately, avoiding the inconvenience of users waiting for the response and improving the experience. The reading position at the pause time is accurately recorded to ensure the accuracy of the bookmarking. By analyzing the reading text, the content of interest is automatically identified, reducing the trouble of manual search and marking and improving efficiency. The sentence definition helps to analyze the semantics and structure of the text, making the bookmarking more intelligent and meaningful, and easier to review and identify the marked content. Determining the text marking possibility according to the sentence nature makes the provided tags closer to the reading needs and improves the practicality and relevance of the tags. Provide a variety of tag types for users to choose, and choose the appropriate tag according to their own needs and preferences to increase the autonomy and personalized experience of users.

[0042] Figure 1 A schematic diagram of an application scenario provided by the present application is provided. When visually impaired people are marking online reading, the method provided by the present application is applied. Specifically, the method provided by the present application is applied to any server, and the server interacts with the mobile device of the visually impaired people. By capturing the pause signal of the mobile device of the visually impaired people, the operation is responded to immediately, avoiding the inconvenience of the user waiting for the response and improving the experience. The reading position at the moment of pause is accurately recorded to ensure the accuracy of the bookmark. By analyzing the reading text, the content of interest is automatically identified, reducing the trouble of manual search and marking, and improving efficiency. Sentence definition helps to analyze the semantics and structure of the text, making bookmarks more intelligent and meaningful, and easier to review and identify the marked content. Determine the possibility of text marking according to sentence type, so that the provided tags are closer to reading needs, and improve the practicality and relevance of tags. Provide a variety of tag types for users to choose from, and choose appropriate tags according to their own needs and preferences to increase the autonomy and personalized experience of users. The specific implementation method can refer to the following embodiments.

[0043] Figure 2 This is a flowchart of a multi-modal assisted bookmarking method for online reading for visually impaired people provided in an embodiment of the present application. The method of this embodiment can be applied to the server in the above scenario. Figure 2 As shown, the method includes: S201, obtaining a pause signal, and determining text information corresponding to the pause time based on the pause signal; The pause signal can be a pause instruction issued by the visually impaired during the reading process.

[0044] The pause moment may be the precise point in time when the visually impaired person issues a pause instruction.

[0045] The text information may be text content such as words, pictures, audio, etc. corresponding to the pause time.

[0046] Specifically, when the visually impaired pause during reading, a pause signal is captured. This can usually be achieved through the pause function of the screen reader, gesture operation of the touch screen, or voice command. The text information corresponding to the moment of pause is determined based on the moment of pause.

[0047] S202, analyzing and reading the text according to the text information, and determining the marking range; The reading text may be text content such as books, articles, web pages, etc. that the visually impaired people are reading.

[0048] The marking range may be the length of text that a visually impaired person may want to bookmark, corresponding to the pause signal issued by the person.

[0049] Specifically, the text information corresponding to the pause moment is analyzed, and a suitable marking range is determined based on factors such as the reading speed of the visually impaired, historical reading data, etc. This range is a paragraph of text, a part of text, or a single sentence of text, depending on the reading behavior of the visually impaired and the characteristics of the text content.

[0050] S203, defining the sentence type of the content within the marked range, obtaining the sentence type of a single sentence in the marked range, and determining the possibility of text marking according to the sentence type; A single sentence text can be a complete sentence within the marked range or a single sentence after being divided into sentences according to punctuation marks.

[0051] Sentence types can be declarative, interrogative, exclamatory, imperative, etc.

[0052] Text tags may contain important information, interesting content, or other text that is worthy of attention within the tag range.

[0053] Specifically, the sentence type is defined for each sentence in the tagging range, and the grammatical structure and semantic content of the sentence, such as declarative sentence, interrogative sentence or exclamatory sentence, are identified. Based on the sentence type definition, it is further determined which types of text tagging is possible.

[0054] S204. According to the text mark, several label types may be designed for the user to choose.

[0055] The tag type can be a type used to classify and mark text. In a specific implementation, the tags can be classified according to each tag, or they can be classified according to specific sentence types. Therefore, a tag classification rule can be pre-designed to facilitate the classification of the currently analyzed text.

[0056] Specifically, a series of tags are designed based on the text tagging possibilities for the visually impaired to choose from. These tags can be predefined or dynamically generated based on the user's historical tagging habits and preferences.

[0057] Through this solution, by capturing the pause signal of the visually impaired, the operation is responded to immediately, avoiding the inconvenience of waiting for the response and improving the user experience. The reading position at the moment of pause is accurately recorded to ensure the accuracy of bookmarking. By analyzing the reading text, the content of interest is automatically identified, reducing the trouble of manual search and marking and improving efficiency. Sentence definition helps to analyze the semantics and structure of the text, making bookmarking more intelligent and meaningful, and easier to review and identify the marked content. Determining the possibility of text tagging based on sentence type makes the provided tags closer to reading needs and improves the practicality and relevance of tags. Provide a variety of tag types for users to choose from, and choose appropriate tags according to their needs and preferences to increase user autonomy and personalized experience.

[0058] In some embodiments, the reading speed before the pause moment is obtained; the text reading length per unit time is determined based on the reading speed; the text reading length and text information are analyzed, the reading text is analyzed, and the marking range is determined.

[0059] The reading speed can be the text reading speed that the visually impaired person actively sets during the reading process.

[0060] The text reading length may be the amount of text that can be read in a unit of time.

[0061] Specifically, the text reading speed actively set by the visually impaired during the reading process is obtained from the reading history records or real-time monitoring. According to the reading speed, the length of the text that can be read per unit time is calculated. The text information such as the text content, paragraph structure, keywords, etc. at the pause time is analyzed to identify the content of interest. The calculated text reading length per unit time is combined with the text information at the pause time to analyze the text content read before the pause time. Based on the reading speed, the text information at the pause time and the text reading length, a marking range that may be planned to be marked by the visually impaired is determined.

[0062] Through this solution, by obtaining the reading speed, the reading habits and rhythm can be identified, so as to better predict the text content that has been read before the pause moment. Calculating the text reading length per unit time helps to determine a rough text range, which is the content read before the pause moment. By analyzing the text reading length and text information, the content of interest can be more accurately identified, providing a basis for determining the marking range. Analyzing the reading text helps to identify the semantics and structure of the text, so as to more accurately determine the part of the text that is of interest. Based on the analysis results, a suitable marking range is determined, which can be the full text, partial text or single sentence text, which helps to bookmark more effectively.

[0063] In some embodiments, device movement data is obtained and analyzed to determine the device picking time; the user's operation time is determined based on the device picking time and the pause time; the text reading record within the operation time is determined based on the text reading length; and the marking range is determined based on the text information and the text reading record.

[0064] Device movement data may be the movement information of a device such as a smartphone or tablet computer during use by the user, which may be detected by a built-in gyroscope.

[0065] The device picking time may be the moment when the device is picked up before the pause signal is issued. If the person using the device continues to hold the device before the pause signal is issued, the moment when the device moves quickly may be used as the device picking time.

[0066] The operation duration may be the length of time from when the device is picked up to when reading is paused (ie, a pause signal is generated).

[0067] The text reading record may be a historical record of the text content read within the operation duration.

[0068] Specifically, the device movement data is collected through sensors or gyroscopes in the device. The device movement data is analyzed to identify the transition point from stationary to moving, that is, the time point when the device is picked up. The time difference between the time the device is picked up and the pause time is calculated to obtain the user's operation time. According to the reading speed and operation time set by the user, the length of the text read during this period is calculated. From the text position at the pause time, the content of the read text is determined based on the calculated text length. The determined text reading records are subjected to syntactic, semantic and other text analysis. According to the text information and reading records, a suitable marking range is determined.

[0069] This solution obtains device movement data and identifies the user's behavior before picking up the device, which helps determine the time when the user starts reading. By analyzing the device movement data, the device pick-up time can be more accurately determined, thereby more accurately calculating the reading time. Combining the device pick-up time and the pause time, the operation time from picking up the device to pausing the operation is calculated. The text content read during this period is determined by the reading speed and operation time. Combining the user's reading record and text information, the content of interest to the user can be more accurately identified, thereby determining a suitable marking range.

[0070] In some embodiments, for each single sentence, the text sentiment of the single sentence is determined based on the sentence type; the text reading record and text sentiment are analyzed to determine the strength of the sentiment connection between the single sentence and the context; the strength of the content connection between the single sentence and the context is determined based on the sentence type; and the possibility of text marking is determined based on the sentiment connection strength and the content connection strength.

[0071] Text sentiment can be the emotional tendency or emotional color expressed by the text content.

[0072] The emotional connection strength can be the emotional closeness between a single sentence text and the context text.

[0073] The content connection strength can be the degree of relevance between a single sentence text and the context text in terms of content.

[0074] Specifically, based on the sentence definition, sentiment analysis is performed on the single sentence text to determine the positive, negative, neutral and other sentiment tendencies. Check whether the sentiment of the single sentence text is consistent with the context. Identify whether the single sentence text forms a transition with the context in sentiment, and then analyze the strength of the sentiment connection between the single sentence text and the context through these judgment results. Based on the sentence definition, analyze the strength of the content connection between the single sentence text and the context. Comprehensively analyze the sentiment connection strength and the content connection strength to determine the possibility of marking the single sentence text. Based on the comprehensive analysis results, determine whether the single sentence text has the possibility of marking.

[0075] Through this scheme, through sentence definition, the basic functions of each sentence such as statement, question or expression of emotion are identified, which is helpful for sentiment analysis and judgment of content connection strength. Text sentiment analysis can reveal the emotional tendency of a single sentence text, such as positive, negative or neutral, which helps to identify the emotional response to the text. By analyzing the emotional connection strength between a single sentence text and the context, the emotional turning point or consistency in the text is identified, so as to better analyze the emotional changes in the reading process. The analysis of content connection strength helps to judge the degree of relevance between a single sentence text and the context in terms of topic and content, and helps to determine the coherence of the text and the user's interest points. Comprehensive analysis can provide a comprehensive perspective, which helps to evaluate the closeness of a single sentence text to the context in terms of emotion and content, so as to more accurately predict the tagging intention. The final determination of text tagging may ensure that the bookmark tagging suggestions match the actual reading interests and emotional responses, thereby improving the accuracy of tagging and user satisfaction.

[0076] In some embodiments, based on the content connection strength, it is determined whether there are at least two single-sentence texts with an inseparable semantic connection; if so, the at least two single-sentence texts with an inseparable semantic connection are regarded as a possible text mark; if not, based on the emotional connection strength, the emotional coherence between the single-sentence text and the remaining text is determined; based on the emotional coherence, the text mark is determined to be possible.

[0077] Inseparability can mean that in text processing, the text elements of two or more sentences or paragraphs are closely connected semantically and cannot be separated without affecting the integrity of the overall meaning.

[0078] Sentence connection can be the connection between two or more sentences in meaning and logic.

[0079] Emotional coherence can be the consistency or logical continuity of emotional expressions in a text between sentences or paragraphs.

[0080] Specifically, keyword matching, topic relevance analysis and other operations are performed on at least two single-sentence texts to determine whether there is an inseparable semantic connection between them. If the analysis results show that there is an inseparable semantic connection between at least two single-sentence texts, these texts are considered as a whole for marking possibility. If there is an inseparable semantic connection, at least two single-sentence texts may be marked as a text. If the analysis results show that there is no inseparable semantic connection, the emotional coherence between the single-sentence text and the remaining texts is analyzed based on the emotional connection strength. Based on the emotional coherence analysis results, it is determined whether the single-sentence text is emotionally consistent with the context text. If consistent, several texts with consistent emotions may be marked as a text.

[0081] Through this scheme, by analyzing the content connection strength between at least two single-sentence texts, the coherence and logical relationship in the text are identified, which helps to identify points of interest and information needs during the reading process. If there is an inseparable sentence connection, these texts are considered as a whole for marking, which helps to better preserve and identify related information content. Marking texts with inseparable sentence connections as bookmarks helps to quickly review and access important information and improve reading efficiency. Analyzing the emotional coherence between a single sentence text and the rest of the text helps to identify emotional changes and reactions during the reading process, thereby providing a more personalized reading experience. By determining emotional coherence, identifying emotional turning points or consistency in the text helps to better analyze the text content and emotional background. Based on the results of the emotional coherence analysis, the marking intention is more accurately predicted, thereby providing more precise and personalized bookmarking suggestions.

[0082] In some embodiments, the reading text is analyzed to determine the text attributes; based on the text attributes, the content type within the marking range is determined; based on the content type, the pause signal is analyzed to determine whether the pause signal is a bookmark establishment signal; if so, the text marking possibility is determined based on the sentence nature.

[0083] Text attributes can be basic features and structural information such as the length of the text content, sentence structure, vocabulary usage, topic distribution, and sentiment tendency.

[0084] Content type can be a classification of content in a text according to its nature and function.

[0085] The bookmarking signal may be a signal indicating an intention during reading that the location is to be marked as a bookmark.

[0086] Specifically, obtain the content of the reading text. Perform preprocessing such as word segmentation and part-of-speech tagging on the text. Analyze the text's attributes such as sentence type, keywords, and themes. Identify the text position at the pause moment. According to the results of the text attribute analysis, determine the content type of the text within the marked range, such as descriptive content, explanatory content, and opinion content. According to the content type and the pause signal, determine whether the pause signal indicates the intention of bookmarking. If it is determined that the pause signal is a bookmark establishment signal, further analyze the text according to the sentence definition. Combined with the sentence analysis results, determine the possibility of text marking.

[0087] Through this solution, the structural and content characteristics of the text are obtained by analyzing the reading text. According to the results of the text attribute analysis, the content types within the marked range, such as descriptive content, illustrative content, opinion content, etc., are identified, which helps to identify the information categories that the user is interested in. By analyzing the pause signal, it is determined whether the intention is to create a bookmark. It helps to provide more accurate marking suggestions to meet the needs of the reading process. If it is determined that the pause signal is a bookmark establishment signal, the text is further analyzed according to the sentence type to provide the user with more specific marking suggestions. It helps to quickly locate and review important information content.

[0088] In some embodiments, the text completeness is determined based on the text reading record; the text completeness is compared with a preset judgment threshold; if the text completeness is lower than the preset judgment threshold, the text reading record is analyzed to determine whether the sentence structure is accurate; if accurate, supplementary text is obtained based on punctuation marks; and the marking range is determined based on the text information, the supplementary text and the text reading record.

[0089] Text integrity can be the result of judging whether the currently read content has been read completely. For example, the above text reading record is determined only by time. At this time, there may be only half a sentence. But in fact, when marking, a whole sentence or a whole paragraph is generally marked, so text integrity needs to be considered.

[0090] The preset judgment threshold may be a preset standard or critical value for evaluating the degree of text integrity, which is pre-stored in the server and called when used.

[0091] Punctuation marks can be symbols such as periods, commas, question marks, etc. in text.

[0092] The supplementary text may be the missing text portion determined above, inferred based on the reading record and punctuation marks in the text.

[0093] Specifically, all the text content that the user is reading is obtained, and the text reading records determined above are analyzed to determine whether there are text omissions in the current text reading record, that is, the situation where it is not a whole paragraph or a whole sentence, and then the text completeness of the text reading record is calculated. The text completeness index is compared with the preset judgment threshold. If the text completeness is lower than the preset threshold, it can be considered that there are omissions. At this time, natural language processing technology is used to analyze the sentence structure in the text reading record to determine whether the sentence structure in the text is correct. If correct, the punctuation in the text is analyzed to determine the sentence boundaries. Based on the punctuation and context information, supplementary text is obtained from all text content to complete the reading record. Combine the text information, supplementary text and reading record to determine the text range you want to mark. According to the pause signal and reading behavior, the text range to be marked is judged.

[0094] This solution analyzes the text reading record to determine whether all parts of the text have been read, thereby ensuring a holistic analysis of the text. Comparing the text completeness index with the preset judgment threshold helps to quickly identify whether the user's reading has reached the expected completeness, thereby determining whether further analysis is needed. Analyzing sentence accuracy helps to identify whether the reading pattern matches the syntactic structure of the text, and helps to improve the recognition of reading intent. Through punctuation analysis, unread or omitted parts of the text are inferred, thereby supplementing the reading record and improving the completeness of the record. Combining text information, supplementary text, and reading records to more accurately determine the range of text you want to mark helps to quickly locate and review content of interest.

[0095] In some embodiments, the user's historical reading records are obtained; the historical reading records are analyzed to determine historical marks and historical pause signals; the historical pause signals and corresponding historical marks are analyzed to determine the user's historical access time; the historical reading settings are obtained; based on the historical reading settings, the historical reading speed corresponding to each historical pause signal is determined; based on the historical reading speed and the historical access time, the historical reading record of the historical operation time corresponding to each historical pause signal is determined; based on the historical reading records and the corresponding historical marks, the user's reading reaction time is determined; based on the text reading length, the reaction reading record within the reading reaction time is determined; based on the text information, supplementary text, reaction reading record and text reading record, the marking range is determined.

[0096] The historical reading records may be reading behavior data such as reading text, reading time, and pause points generated when using reading tools in the past.

[0097] History marks can be bookmarks, highlights, annotations, and other marks made to text content during the historical reading process.

[0098] The history pause signal may be a signal for pausing during the history reading process.

[0099] The history fetching time may be a time point at which the history reading process is paused.

[0100] The historical reading settings may be personalized parameters such as reading speed, font size, background color, etc. set during historical reading.

[0101] The historical reading speed may be a reading speed set by the user during the historical reading process.

[0102] The history operation duration can be compared to the above operation duration, which is the length of time spent pausing during the history reading process.

[0103] Reading reaction time can be the time to think, mark, etc. after reading a text.

[0104] Reaction reading records can be reading records caused by reading speed during reading reaction time. This is caused by the reading without pausing from the time when you react to mark to the time when you pause.

[0105] Specifically, retrieve the user's historical reading records from the database. Analyze the historical reading records to identify the historical marks and historical pause signals made previously. Combine the historical pause signals and historical marks to determine the historical access time in the historical reading. Analyze the historical reading settings to determine the reading speed at each historical pause signal, that is, the historical reading speed. According to the historical reading speed and the historical access time, use mathematical analysis methods to calculate the historical reading records under the historical operation time corresponding to each historical pause signal. Analyze the historical reading records and the corresponding historical marks to determine the time difference between the moment when the user takes action when reacting to marking and the time when reading to the mark, that is, the reading reaction time. According to the reading length of the text and the reading reaction time, determine the reaction reading record within the reading reaction time. Finally, combine the text information, supplementary text, reaction reading record and the overall text reading record to determine the text range to be marked.

[0106] Through this solution, historical reading data is collected to provide basic information for analyzing reading habits and preferences. By analyzing historical data, identifying marking behaviors and pause points during reading helps analyze reading patterns and points of interest. Determining the specific time points of pauses in historical reading helps calculate reading speed and reaction time. Identifying personalized settings in historical reading, such as reading speed, helps analyze reading behavior more accurately. Calculating the reading speed at each pause helps infer the degree of concentration when reading text. Determining the reading duration at each pause helps identify behavioral patterns during reading. Determining the reaction time to text content through historical data analysis helps identify the thought process when reading. Determining the text content read within the reaction time helps identify changes in interest during reading. Combining text content, supplementary text, and reading behavior to determine the range of text you want to mark helps improve the accuracy and satisfaction of bookmarking.

[0107] In some embodiments, the sentiment relevance between a single sentence and the context is quantified through the sentiment word strength matrix and the context attention weight. The inter-sentence logical relevance of each adjacent single sentence is modeled through the dependency syntax tree and latent semantic analysis. Based on the sentiment relevance and the inter-sentence logical relevance, a joint matrix is ​​established to determine the content connection strength between the single sentence and the context.

[0108] The sentiment word intensity matrix may be a mathematical matrix containing records of each sentiment word in the text and its intensity.

[0109] Contextual attention weights can be the different degrees of importance given to different parts of the context when processing text.

[0110] Sentiment relevance can be the correlation between the sentiment content in the text and the context.

[0111] A dependency syntax tree can be a representation of syntactic analysis that shows the dependency relationship between words in a sentence.

[0112] Latent semantic analysis may be a statistical method for analyzing the semantic content of text.

[0113] The logical correlation between sentences can be the logical relationship such as cause and effect, transition, progression, etc. between adjacent sentences.

[0114] The joint matrix can be a matrix that integrates multiple features such as sentiment relevance and inter-sentence logical relevance.

[0115] Specifically, perform word segmentation, part-of-speech tagging, and syntactic analysis on the text. Identify sentiment words in the text and construct a sentiment word strength matrix. Use the sentiment word strength matrix to quantify the sentiment strength in a single sentence. According to the context attention weight, adjust the sentiment strength of a single sentence to reflect its sentiment relevance to the context. Construct a dependency syntax tree to analyze the grammatical structure relationship between sentences. Perform latent semantic analysis to extract the semantic association between sentences. Combine sentiment relevance with inter-sentence logical relevance to establish a joint matrix. Determine the content connection strength between a single sentence and the context through matrix operations. Analyze the data in the joint matrix to determine the content connection strength between a single sentence and the context.

[0116] In the specific implementation, the sentiment relevance between a single sentence and the context is quantified through the sentiment word intensity matrix and the context attention weight; as follows: 1. Construct the emotional feature matrix: Suppose a single sentence The sentiment words, negative words, and degree words vectors are , based on the pre-set sentiment word library and correction rules (such as the weight adjustment of modifiers), expression (1) is generated: (1) in, Represents a single sentence Emotional words; Represents a single sentence Negative word of Represents a single sentence The degree word vector of , , Respectively represent the number of sentiment words, the number of negative words, and the number of degree word vectors; 2. Construct the contextual attention weight matrix: The hidden state is extracted using a bidirectional GRU, expressed as follows: (2) in, Indicates hidden state; Neural network regression prediction algorithm; The weight is calculated through the attention mechanism and is expressed by the following expression (3): (3) in, Indicates Column and row element; represents the normalized exponential function; Represents the first Elements of a column; represents a learnable parameter; Represents the first Transpose of the elements of a column; represents the scaling factor; 3. Combined with the logarithmic adjustment formula, determine the single sentence With context The emotional connection strength is expressed by formula (4): (4) in, is the emotional connection strength function; Indicates Column and Elements of a row; is the penalty coefficient for emotional fluctuation; Indicates sentiment words; is the variance of the contextual sentiment vector (reflecting sentiment coherence).

[0117] The logical association between sentences is modeled through dependency syntactic tree and latent semantic analysis (LSA); as follows: 1. Based on the dependency distance metric, construct the sentence dependency matrix : (5) in, Indicates the dependency correlation between sentences; Represents the shortest path length of the core verb in the dependency tree of two sentences (the larger the dependency distance, the weaker the association). Expressing sentences Related sentences of; 2. Latent semantic matrix after dimensionality reduction using SVD: (6) in, represents the left singular vector in the singular value decomposition; represents a diagonal matrix where the elements on the diagonal are singular values; represents the transposed matrix of right singular vectors; Represents the weight matrix of terms in the document; Before interception Dimensional semantic space , calculate the cosine similarity: (7) in, is the cosine similarity function in the latent semantic space; is the current sentence to be analyzed; For Sentences Related sentences of; For Sentences Embedded representation in the reduced k-dimensional latent semantic space; For Sentences Embedded representation of ; 3. Synthesize syntactic and semantic associations: (8) in, Representing quantized sentences With context Algorithms for content relevance; represents the syntactic weight coefficient; Indicates the current sentence The context collection of It represents the dependency relationship between sentences.

[0118] The sentence groups to be marked are screened through the sentiment-content joint matrix; as follows: 1. For each sentence , calculate the comprehensive score: (9) in, Expressing sentences a weighted composite score on emotional intensity and content relevance; is the emotion-content balance factor (usually 0.5-0.7); Indicates the strength of emotional connection; Indicates the current sentence The context collection of Indicates content relevance.

[0119] 2. Set adaptive threshold according to the overall distribution of context: (10) in, Representation function: Dynamically calculate the comprehensive score of the sentence Adaptive threshold Adaptive threshold; represents the mean comprehensive score, represents the standard deviation of the comprehensive score, Indicates the sensitivity parameter (usually ).

[0120] 3. If Make and , it is marked as an inseparable sentence group.

[0121] Otherwise, if , it is marked as an independent key sentence. Through this solution, the structure and content of the text can be more accurately identified through word segmentation and part-of-speech tagging. The sentiment word strength matrix helps to identify and quantify the sentiment tendency in the text, and the context attention weight ensures that the results of sentiment analysis are closer to actual feelings and reading interests. Dependency syntactic tree analysis provides grammatical structure information within the sentence, and latent semantic analysis reveals the semantic relationship between sentences. The two together provide a basis for analyzing the logical correlation between sentences. The joint matrix combines sentiment correlation and inter-sentence logical correlation to provide a comprehensive evaluation index for the content connection between a single sentence text and the context. By analyzing the joint matrix, the content connection strength between a single sentence text and the context is determined, and accurate bookmarking suggestions are provided, which helps users manage and collect reading content more effectively.

[0122] Figure 3 A schematic diagram of a bookmarking system for multimodal online reading assistance for visually impaired people provided in one embodiment of the present application is shown in FIG. Figure 3 As shown, the bookmarking system 300 for multimodal assistance in online reading for the visually impaired in this embodiment includes: an information determination module 301 , a range determination module 302 , a possibility analysis module 303 , and a label design module 304 .

[0123] The information determination module 301 is used to obtain a pause signal and determine text information corresponding to the pause moment based on the pause signal; A range determination module 302 is used to analyze the reading text and determine the marking range according to the text information; Possibility analysis module 303, used to define the sentence type of the content within the marked range, obtain the sentence type of a single sentence text within the marked range, and determine the possibility of text marking according to the sentence type; The label design module 304 is used to design several label types according to the text mark for the user to choose.

[0124] Optionally, when the range determination module 302 analyzes and reads the text according to the text information and determines the marking range, it is used to: Get the reading speed before the pause moment; Determining the text reading length per unit time according to the reading speed; Analyze the text reading length and the text information, analyze the reading text, and determine the marking range.

[0125] Optionally, the range determination module 302 analyzes the text reading length and the text information, analyzes the reading text, and determines the marking range, for: Acquire device movement data, analyze the device movement data, and determine the device pick-up time; Determining the operation time of the user according to the device picking time and the pause time; Determining the text reading record within the operation time according to the text reading length; A marking range is determined according to the text information and the text reading record.

[0126] Optionally, when the possibility analysis module 303 determines that the text tag is possible according to the sentence type, it is used to: For each single sentence text, determining the text sentiment of the single sentence text according to the sentence type; Analyzing the text reading record and the text sentiment to determine the sentiment connection strength between the single sentence text and the context; Determining the content connection strength between the single sentence text and the context according to the sentence type; The possibility of text tagging is determined according to the emotional connection strength and the content connection strength.

[0127] Optionally, when the possibility analysis module 303 determines that the text tag is possible according to the sentiment connection strength and the content connection strength, it is used to: According to the content connection strength, determining whether there are at least two single-sentence texts that have inseparable sentence meaning connections; If so, it is possible to mark the at least two single-sentence texts that are inseparable from each other as a text; If not, determining the emotional coherence between the single sentence text and the rest of the text according to the emotional connection strength; Based on the sentiment coherence, a text tagging possibility is determined.

[0128] Optionally, when the possibility analysis module 303 determines that the text tag is possible according to the sentence type, it is used to: Analyzing the reading text to determine text attributes; Determining the content type within the marked range according to the text attribute; Analyzing the pause signal according to the content type to determine whether the pause signal is a bookmark establishment signal; If so, the text tagging possibility is determined based on the sentence type.

[0129] Optionally, when determining the marking range according to the text information and the text reading record, the range determination module 302 is used to: Reading records based on the text to determine the integrity of the text; Comparing the text integrity with a preset judgment threshold, and if the text integrity is lower than the preset judgment threshold, analyzing the text reading record to determine whether the sentence structure is correct; If it is accurate, then based on the punctuation, a supplementary text is obtained; A marking range is determined according to the text information, the supplementary text and the text reading record.

[0130] Optionally, when the range determination module 302 determines the marking range according to the text information, the supplementary text and the text reading record, it is used to: Obtain the user's historical reading records; Analyze the historical reading records to determine historical marks and historical pause signals; Analyze the historical pause signal and the corresponding historical mark to determine the historical picking time of the user; Acquire historical reading settings; determine the historical reading speed corresponding to each historical pause signal according to the historical reading settings; Determine a historical reading record of the historical operation duration corresponding to each historical pause signal according to the historical reading speed and the historical fetching time; Determining the reading reaction time of the user according to the historical reading records and the corresponding historical marks; Determine the reaction reading record within the reading reaction time according to the text reading length; A marking range is determined according to the text information, the supplementary text, the reaction reading record and the text reading record.

[0131] Optionally, when the possible analysis module 303 determines the content connection strength between the single sentence text and the context according to the sentence type, it is used to: The sentiment relevance between the single sentence text and the context is quantified through the sentiment word intensity matrix and the context attention weight.

[0132] Through dependency syntactic tree and latent semantic analysis, the logical correlation between each adjacent sentence text is modeled.

[0133] According to the sentiment relevance and the inter-sentence logical relevance, a joint matrix is ​​established to determine the content connection strength between the single sentence text and the context.

[0134] The system of this embodiment can be used to execute the method of any of the above embodiments. The implementation principles and technical effects are similar and will not be described in detail here.

Claims

1. A multimodal assisted bookmarking method for online reading for visually impaired people, characterized in that: include: Obtaining a pause signal, and determining text information corresponding to the pause moment based on the pause signal; Analyze and read the text according to the text information to determine the marking range; Defining the sentence type of the content within the marked range, obtaining the sentence type of a single sentence within the marked range, and determining the possibility of text marking according to the sentence type; Several label types may be designed based on the text mark for the user to choose from.

2. The method according to claim 1, characterized in that The step of analyzing and reading the text and determining the marking range according to the text information includes: Get the reading speed before the pause; Determining the text reading length per unit time according to the reading speed; Analyze the text reading length and the text information, analyze the reading text, and determine the marking range.

3. The method according to claim 2, characterized in that The analyzing the text reading length and the text information, analyzing the reading text, and determining the marking range includes: Acquire device movement data, analyze the device movement data, and determine the device pick-up time; Determining the operation time of the user according to the device picking time and the pause time; Determining the text reading record within the operation time according to the text reading length; A marking range is determined according to the text information and the text reading record.

4. The method according to claim 2, characterized in that: Determining the text tag according to the sentence type may include: For each single sentence text, determining the text sentiment of the single sentence text according to the sentence type; Analyzing the text reading record and the text sentiment to determine the sentiment connection strength between the single sentence text and the context; Determining the content connection strength between the single sentence text and the context according to the sentence type; The possibility of text tagging is determined according to the emotional connection strength and the content connection strength.

5. The method according to claim 4, characterized in that The determining of the text tag possibility according to the emotional connection strength and the content connection strength includes: According to the content connection strength, determining whether there are at least two single-sentence texts that have inseparable sentence meaning connections; If so, it is possible to mark the at least two single-sentence texts that are inseparable from each other as a text; If not, determining the emotional coherence between the single sentence text and the rest of the text according to the emotional connection strength; Based on the sentiment coherence, a text tagging possibility is determined.

6. The method according to claim 5, characterized in that Determining the text tag according to the sentence type may include: Analyzing the reading text to determine text attributes; Determining the content type within the marked range according to the text attribute; Analyzing the pause signal according to the content type to determine whether the pause signal is a bookmark establishment signal; If so, the text tagging possibility is determined based on the sentence type.

7. The method according to claim 3, characterized in that The step of determining the marking range according to the text information and the text reading record includes: Reading records based on the text to determine the integrity of the text; Comparing the text integrity with a preset judgment threshold, and if the text integrity is lower than the preset judgment threshold, analyzing the text reading record to determine whether the sentence structure is correct; If it is accurate, then based on the punctuation, a supplementary text is obtained; A marking range is determined according to the text information, the supplementary text and the text reading record.

8. The method according to claim 7, characterized in that The step of determining the marking range according to the text information, the supplementary text and the text reading record includes: Obtain the user's historical reading records; Analyze the historical reading records to determine historical marks and historical pause signals; Analyze the historical pause signal and the corresponding historical mark to determine the historical picking time of the user; Acquire historical reading settings; determine the historical reading speed corresponding to each historical pause signal according to the historical reading settings; Determine a historical reading record of the historical operation duration corresponding to each historical pause signal according to the historical reading speed and the historical fetching time; Determining the reading reaction time of the user according to the historical reading records and the corresponding historical marks; Determine the reaction reading record within the reading reaction time according to the text reading length; A marking range is determined according to the text information, the supplementary text, the reaction reading record and the text reading record.

9. The method according to claim 1, characterized in that: Determining the content connection strength between the single sentence text and the context according to the sentence type includes: quantifying the sentiment relevance between the single sentence text and the context through the sentiment word intensity matrix and the context attention weight; Through dependency syntactic tree and latent semantic analysis, the logical correlation between each adjacent sentence is modeled; According to the sentiment relevance and the inter-sentence logical relevance, a joint matrix is ​​established to determine the content connection strength between the single sentence text and the context.

10. A multimodal bookmarking system for online reading for the visually impaired, characterized in that: The method as claimed in any one of claims 1 to 9 comprises: An information determination module, used for obtaining a pause signal, and determining text information corresponding to the pause moment based on the pause signal; A range determination module, used to analyze and read the text and determine the marking range according to the text information; A possibility analysis module is used to define the sentence type of the content within the marked range, obtain the sentence type of a single sentence text within the marked range, and determine the possibility of text marking according to the sentence type; The label design module is used to design several label types according to the text mark for users to choose.

Images