Voice text dynamic highlighting and intelligent positioning assembly based on ReactNative

Abstract

The invention is applicable to the field of mobile application development, and provides a ReactNative-based voice text dynamic highlighting and intelligent positioning component, which is characterized in that a timestamp-position mapping index is pre-calculated and established by initializing a layout, and fragmented contents are integrated by combining a semantic-driven text merging algorithm, so as to realize dynamic highlighting and intelligent positioning of a voice text. And a three-state interaction control model is constructed to realize state switching of automatic following and manual browsing. The component solves the problems of low real-time calculation performance, fragmented display visual interference, interaction logic conflict and the like in the existing scheme, is suitable for cross-platform mobile applications such as online education and voice notes, and improves the positioning efficiency, reading experience and operation convenience and reduces the multi-terminal development cost through pre-calculation optimization, intelligent combination and intelligent interaction.

Description

Technical Field

[0001] The present invention belongs to the field of mobile application development, and in particular relates to a React Native-based voice-to-text dynamic highlighting and intelligent positioning component. Background Art

[0002] In mobile app development, the synchronized voice and text display feature is widely used in scenarios such as online education, voice notes, and audiobook reading. As users' expectations for interactive experiences increase, achieving real-time, synchronized highlighting and precise positioning of voice playback and text content has become a key requirement. This technology must balance performance efficiency, display quality, and interactive fluidity to adapt to different devices and usage scenarios.

[0003] Existing React Native solutions generally use a real-time positioning calculation technology path, dynamically calculating text position by monitoring audio playback events. During audio playback, the layout of the Text component is triggered in real time to determine the text position, and the fragmented text output by the speech transcription tool (averaging 1-2 words / segment) is directly rendered. This does not distinguish between program-controlled scrolling and user-controlled scrolling. As a result, existing React Native solutions have low real-time layout calculation efficiency. Each positioning triggers the reflow of the Text component, which can easily cause page lag on low-end devices. The direct rendering of fragmented text causes frequent highlight switching and visual interference. The scrolling operation of the program and the user is not distinguished, and the coordinate system conflicts cause interface anomalies. Summary of the Invention

[0004] The purpose of the present invention is to provide a React Native-based voice text dynamic highlighting and intelligent positioning component, aiming to solve the technical problems existing in the prior art identified in the background technology.

[0005] The present invention is implemented as follows: a React Native-based voice text dynamic highlighting and intelligent positioning component, the component comprising:

[0006] Data preprocessing module

[0007] This function groups fragmented text by paragraph ID, merges it according to natural language segmentation rules, and verifies the grammatical integrity of the merged text. Natural language segmentation rules include: forcing text segments containing periods, question marks, and exclamation marks to be merged into independent text blocks; allowing cross-segment merging of text segments containing commas, semicolons, and colons and with a single sentence of 20 words or less; and merging text segments without punctuation at intervals greater than 500ms.

[0008] Pre-calculation module

[0009] The full text scrolling height is cached through the onLayout event of the Text component, and a timestamp-position mapping index is established. Each index item in the index contains the offset offset, height height and corresponding audio time interval timeRange of the text block.

[0010] Dynamic positioning module

[0011] Based on the precomputed timestamp-position mapping index, a binary search algorithm is used to perform a fast, time-complex mapping of playback time to text position. The binary search algorithm logic is as follows: the left boundary is initialized to 0 and the right boundary is initialized to the length of the precomputed index data minus one. The middle position is then calculated in a loop. If the current playback time falls within the audio time interval of the corresponding index item at the middle position, the index is returned. Otherwise, the boundaries are adjusted until a match is found and -1 is returned.

[0012] Interactive control module

[0013] Detect manual operation through ScrollView scroll events and control the state switching between automatic scrolling and manual operation. The state switching mechanism includes: switching from automatic following state to manual browsing state when receiving the ScrollView scroll start event; displaying control buttons in manual browsing state and resuming automatic following state through seekTo callback when the control button is clicked.

[0014] Exception handling module

[0015] Sort text fragments with out-of-order timestamps in ascending order by the bg field; filter out invalid text fragments with empty content or containing only spaces; use weak references to cache scroll positions and release resources when the component is uninstalled.

[0016] The beneficial effects of the present invention are:

[0017] This component avoids real-time layout calculations through pre-calculation technology, significantly improving scrolling positioning efficiency and increasing the fluency of low-end devices by over 30%. It integrates fragmented text based on natural language rules to make highlighted display consistent with reading habits and reduce visual fatigue. It intelligently switches between automatic and manual modes, improving operational convenience and fault tolerance through visual feedback and adaptive button design. It achieves cross-iOS / Android platform compatibility based on ReactNative, reducing multi-terminal development costs and comprehensively optimizing the performance, display effects and interactive experience of synchronous voice and text display. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is the architecture diagram of the ReactNative-based voice-to-text dynamic highlighting and intelligent positioning component. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] like Figure 1 As shown, a voice text dynamic highlighting and intelligent positioning component based on ReactNative:

[0021] 1. Data Processing and Integration

[0022] 1. Data structure definition

[0023] The voice text data uses a standardized array format. Each element contains: the unique paragraph identifier wp_dl, the user role identifier role, the audio time interval bg / ed (milliseconds), and the text content content.

[0024] 2. Intelligent merging rule implementation

[0025]

[0026]

[0027] 3. Syntax integrity check

[0028] After merging, the text is checked for grammatical integrity. When anomalies such as unclosed quotation marks and incomplete URLs are detected, the display will fall back to the original segment to ensure the accuracy of the text display.

[0029] 2. Pre-calculation and positioning implementation

[0030] 1. Layout pre-calculation process

[0031] During the component initialization phase, the scroll height of all text blocks is calculated and cached at once using the Text component onLayout callback.

[0032] The specific process is: traverse all text references, obtain the position and size information of each text block through the measure method, including the offset, height, and the corresponding audio time interval timeRange, and store this information in the pre-calculated timestamp-position mapping index to provide data support for subsequent positioning.

[0033] 2. Dynamic positioning algorithm implementation

[0034] Based on pre-calculated index data, a binary search algorithm is used to quickly map playback time to text position. The specific steps are as follows:

[0035] 1). Initialize the left boundary to 0 and the right boundary to the length of the pre-calculated index data minus one.

[0036] 2). Loop through the middle positions of the left and right boundaries to determine whether the current playback time is within the audio time interval of the index item corresponding to the middle position.

[0037] 3). If it is within the range, the position of the index item is returned; if the end time of the audio time interval corresponding to the index item at the middle position is earlier than the current playback time, the left boundary is adjusted to the middle position plus one; if the start time of the audio time interval corresponding to the index item at the middle position is later than the current playback time, the right boundary is adjusted to the middle position minus one.

[0038] 4). Repeat the above steps until the target position is matched or -1 is returned, achieving fast positioning with O(logn) time complexity.

[0039] 3. Interaction Control Logic

[0040] 1. State switching mechanism implementation

[0041] The initial state is the automatic follow state (AutoFollow). When the ScrollView scroll start event (onScrollBeginDrag) is received, it switches to the manual browsing state (ManualBrowse).

[0042] In manual browsing mode, a control button is displayed. After the user clicks the control button, the automatic following mode is restored through the seekTo callback (ResetPending→AutoFollow).

[0043] When the component is uninstalled, the manual browsing state returns directly to the initial state.

[0044] 2. Visual feedback design implementation

[0045] The highlight style uses a semi-transparent background color (such as #FFEB3B 20% transparency) to highlight the currently playing text and enhance the user's visual experience.

[0046] The control button is fixed in the lower right corner of the screen, with a size of 44x44pt, a 0.3s fade-in animation, and the text is "Return to audio sync position". The button position automatically adapts to the screen safe area, avoiding areas such as special-shaped screens and the bottom navigation bar.

[0047] 4. Implementation of exception handling module

[0048] 1. Timestamp out-of-order processing

[0049] When out-of-order timestamps are detected, the fragments of the same paragraph are automatically sorted in ascending order by the bg field to ensure that the text display order is consistent with the audio playback order.

[0050] 2. Empty content filtering

[0051] Ignore invalid text fragments, that is, text fragments with empty content or containing only spaces, and do not render them, thereby improving component performance.

[0052] 3. Memory optimization

[0053] Use weak references to cache scroll positions and automatically release resources when the component is uninstalled to avoid memory leaks and ensure efficient operation of the component.

[0054] 5. Cross-platform adaptation

[0055] iOS: Use ScrollView's contentOffset to precisely control scroll position

[0056] Android: Calibrate position by combining scrollTo() method with layout parameters

[0057] Unified interface: encapsulates platform-independent scrolling control functions and automatically handles device pixel ratio (DPR) differences

[0058] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0059] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A ReactNative-based voice text dynamic highlighting and intelligent positioning component, characterized by: The components include: The data preprocessing module is used to group the fragmented texts by paragraph ID and merge them according to natural language segmentation rules, and to perform grammatical integrity check on the merged texts. The pre-calculation module is used to cache the full text scroll height through the Text component onLayout event and establish a timestamp-position mapping index; Dynamic positioning module, which is used to perform fast mapping of playback time to text position using binary search algorithm based on pre-computed timestamp-position mapping index; The interactive control module is used to detect manual operations through ScrollView scroll events and control the state switching between automatic scrolling and manual operations.

2. The assembly according to claim 1, characterized in that The natural language segmentation rules include: Force text fragments containing periods, question marks, and exclamation marks to be merged into independent text blocks; Cross-segment merging is allowed for text segments that contain commas, semicolons, colons, and have a single sentence of ≤20 words; For non-punctuation text segments, the time interval is >500ms and the segments are separated and merged.

3. The assembly according to claim 1, characterized in that In the timestamp-position mapping index, each index item includes the offset offset, height height and corresponding audio time interval timeRange of the text block.

4. The assembly according to claim 1, wherein The binary search algorithm logic is: Initialize the left boundary to 0 and the right boundary to the length of the pre-calculated index data minus one; Loop through the middle position. If the current playback time is within the audio time interval of the index item corresponding to the middle position, return the index. Otherwise, adjust the boundaries until they match and return -1.

5. The assembly according to claim 1, characterized in that The state switching mechanism of the interactive control module includes: When the automatic following state receives the ScrollView scroll start event, it switches to the manual browsing state; The control button is displayed in the manual browsing state, and when the control button is clicked, the seekTo callback is used to restore the automatic following state.

6. The assembly according to claim 1, characterized in that Also includes exception handling module: Arrange text fragments with out-of-order timestamps in ascending order by the bg field; Filter invalid text fragments that are empty or contain only spaces; Use weak references to cache the scroll position and release resources when the component is unmounted.

Images