A data backtracking method, electronic device, and storage medium

Abstract

This application discloses a data backtracking method, electronic device, and storage medium. It includes: determining the current playback time and the element to be backtracked during the event backtracking process; obtaining playback time information corresponding to the elements in the event; classifying the elements to be backtracked into played elements, currently playing elements, and elements to be played based on the current playback time and the playback time information; rendering the played elements and the currently playing elements, and automatically adjusting the view so that the currently playing element plays within the backtracking display area; if the currently playing element finishes playing, controlling the elements in the elements to be played to continue playing based on the playback time information corresponding to the elements in the event, and automatically adjusting the view again so that the continuing playback element plays within the backtracking display area, until the playback of the elements to be backtracked is completed. This application allows backtracking without recording, accurately classifies playback status, dynamically renders elements, and achieves an efficient and intuitive user experience and a smooth backtracking process.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data backtracking method, electronic device, and storage medium. Background Technology

[0002] In current multimedia applications and data backtracking systems, users frequently need to view and analyze past events. These events can include online meetings, online education, and other forms of media content. However, existing multimedia backtracking technologies often rely on audio recordings as the primary means of data backtracking, such as playing recordings to backtrack corresponding handwriting records. If no recordings are made for a certain period, the system will be unable to backtrack the corresponding handwriting information, leading to information loss and a degraded user experience. Summary of the Invention

[0003] One objective of this application is to provide a data backtracking method, electronic device, and storage medium to solve the technical problem that existing technologies cannot backtrack relevant elements in the absence of recording.

[0004] To address the aforementioned technical problems, one technical solution adopted in this application is: providing a data backtracking method, comprising: determining the current playback time and the element to be backtracked during the event backtracking process; obtaining playback time information corresponding to the elements in the event; classifying the element to be backtracked into played elements, currently playing elements, and elements to be played according to the current playback time and the playback time information; rendering the played elements and the currently playing elements, and automatically adjusting the view so that the currently playing element plays within the backtracking display area; if the currently playing element finishes playing, controlling the elements in the elements to be played to continue playing according to the playback time information corresponding to the elements in the event, and automatically adjusting the view again so that the continuing playback element plays within the backtracking display area, until the playback of the elements to be backtracked is completed.

[0005] This method allows users to rewind without recording by acquiring the playback time information of all elements in an event. This solves the problem in existing technologies where handwriting and other elements cannot be viewed due to the lack of recording, ensuring that users can fully understand all relevant content of the event. Furthermore, this method determines the currently playing, already played, and yet-to-be-played elements based on the current playback time and playback time information. This classification ensures a clear understanding of the playback progress and status during rewind, improving the effectiveness of the rewind process. Moreover, rendering the already played and currently playing elements and playing the current element within the rewind display area allows users to intuitively understand the playback status. Finally, it can automatically control playback, improving the convenience and smoothness of operation and ensuring the continuity of the playback process. Therefore, this method, by independently acquiring the playback time information of all elements in an event, allowing rewind without recording, accurately classifying playback status, and dynamically rendering elements, achieves an efficient and intuitive user experience and a smooth rewind process.

[0006] In some embodiments, obtaining the playback time information corresponding to the elements in the event includes: obtaining the time information corresponding to each element in the event; obtaining the playback order and playback time of each element based on the time information; and optimizing the playback time based on the playback order and playback time of each element to obtain optimized playback time information, wherein the optimized playback time information is the playback time information corresponding to the elements in the event. By optimizing the playback time information, not only is the effectiveness of backtracking improved, but a clear event structure and sequence are also provided to the user.

[0007] In some embodiments, optimizing the playback time based on the playback order and playback time of each element to obtain optimized playback time information, wherein the optimized playback time information is the playback time information corresponding to the elements in the event, includes: when it is determined based on the playback order and playback time that the playback time interval between two adjacent elements is greater than a preset time threshold, adjusting the playback time interval between the two adjacent elements to the preset time threshold; adding a preset time interval between the last element and the first element according to the playback order; obtaining the playback time of the element after adjusting the playback time interval between the two adjacent elements and adding the preset time interval between the last element and the first element, wherein the adjusted playback time and the playback order constitute the playback time information corresponding to the elements in the event. Specifically, adjusting the playback time interval between two adjacent elements ensures smooth playback and avoids playback interruptions caused by excessively large time intervals; the added time interval between the last element and the first element ensures a consistent time experience for the user when rewinding.

[0008] In some embodiments, the elements include handwriting, images, documents, and audio recordings. Obtaining the time information corresponding to each element in the event includes: when the element is handwriting, obtaining the start time and end time of the handwriting generated in the event, and the relative time of each point in the handwriting relative to the start time, wherein the start time, the end time, and the relative time constitute the time information of the handwriting; when the element includes images and documents, obtaining the insertion time corresponding to inserting an image and inserting a document in the event, wherein the insertion time constitutes the time information of the image and the document; when the element includes audio recordings, obtaining the start time and end time of recording in the event, wherein the start time and the end time constitute the time information of the audio recording. By defining different methods for obtaining time information for different types of elements, it ensures that each element has a clear and complete time record in the event, improving the accuracy and traceability of the data.

[0009] In some embodiments, classifying the elements to be traced back into played elements, currently playing elements, and elements to be played based on the current playback time and the playback time information includes: obtaining the start time, end time, and playback order of each element based on the playback time information; determining the elements whose start time matches the current playback time as currently playing elements; determining the elements whose end time is less than the current playback time as played elements; determining the elements whose start time is greater than the current playback time as elements to be played; and sorting the currently playing elements, played elements, and elements to be played according to the playback order to obtain sorted currently playing elements, played elements, and elements to be played. This precise management of playback time information and effective element classification and sorting improves the clarity of the playback status and ensures accurate information retrieval.

[0010] In some embodiments, rendering the played element and the currently playing element, and automatically adjusting the view so that the currently playing element plays within the backtracking display area, includes: rendering the played element and the currently playing element to a preset effect; calculating the bounding rectangle of the currently playing element, and automatically adjusting the size of the bounding rectangle so that the size of the bounding rectangle matches the backtracking display area, thereby allowing the currently playing element to play within the backtracking display area. Through the optimization of visual effects and the precise adjustment of the playback area, not only is the user's viewing experience and the visibility of the content improved, but the continuity and dynamic adaptability of the backtracking process are also ensured.

[0011] In some embodiments, calculating the bounding rectangle of the currently playing element and automatically adjusting its size to match the backtracking display area includes: obtaining the position and size of the currently playing element; defining a bounding rectangle that surrounds the currently playing element based on its position and size; if the width and height of the bounding rectangle are greater than the width and height of the backtracking display area, scaling the bounding rectangle proportionally to fit the backtracking display area; if the width and height of the bounding rectangle are less than the minimum width and height of the backtracking display area, enlarging the bounding rectangle proportionally to match the minimum width and height of the backtracking display area. By calculating the bounding rectangle of the currently playing element and scaling or enlarging it appropriately, the system ensures that the element is always visible within the backtracking display area, avoiding content loss or obstruction due to size mismatch, thus enhancing the flexibility and effectiveness of the application.

[0012] In some embodiments, after determining the current playback time and the element to be replayed, the method further includes: setting the element to be replayed to a preset transparent state, and adding a rendering layer on the screen corresponding to the user interface. The rendering layer is used to draw the replay effect of the element to be replayed in subsequent steps. Setting the element to be replayed to semi-transparent helps the user better understand the context of the current replay; the added rendering layer is to ensure that the elements visually stand out and do not interfere with the display of other elements when drawing their replay effects in subsequent steps.

[0013] To address the aforementioned technical problems, one technical solution adopted in this application is to provide an electronic device, including a memory and a processor. The memory is connected to the processor, and the processor is configured to execute one or more computer programs stored in the memory. When executing the one or more computer programs, the processor enables the electronic device to implement a data backtracking method applicable to the electronic device. This electronic device possesses the beneficial effects corresponding to the aforementioned data backtracking method applicable to electronic devices.

[0014] To address the aforementioned technical problems, one technical solution adopted in this application is to provide a non-volatile computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by an electronic device, the electronic device performs the data backtracking method described above. This non-volatile computer-readable storage medium has the same beneficial effects as the data backtracking method described above.

[0015] To address the aforementioned technical problems, one technical solution adopted in this application is to provide a computer program product, comprising a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by an electronic device, cause the electronic device to perform the data backtracking method described above. This computer program product has the same beneficial effects as the aforementioned data backtracking method. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a data backtracking method provided in an embodiment of this application;

[0018] Figure 2 This is a flowchart of a method for obtaining playback time information corresponding to elements in an event, as provided in an embodiment of this application.

[0019] Figure 3 This is a flowchart of a method provided in this application embodiment for rendering played elements and currently playing elements, and automatically adjusting the view so that the currently playing element plays within the backtracking display area;

[0020] Figure 4 This is a schematic diagram of the data backtracking device provided in the embodiments of this application;

[0021] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0023] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, and all are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0024] In current multimedia applications and data backtracking systems, users frequently need to view and analyze past events for learning, review, or evaluation. These events can include various forms of media content such as online meetings, online education, and training courses. However, existing multimedia backtracking technologies often rely on audio recordings as the primary means of data backtracking, with users typically playing recordings to backtrack corresponding handwriting records. While effective in some cases, this method has significant limitations. When no recordings are available for a certain period, the system cannot backtrack the corresponding handwriting information, resulting in users not obtaining complete event information during backtracking, affecting the comprehensiveness and accuracy of the information. Moreover, users must process both audio and handwriting simultaneously during backtracking, increasing the complexity of the operation. Therefore, existing technologies fail to effectively integrate information from handwriting, audio recordings, and other elements, making it difficult for users to obtain comprehensive context during backtracking and affecting the depth of event analysis.

[0025] This application proposes a novel data backtracking method to address the problems of the prior art. The core concept of this method is to allow users to backtrack without recording by acquiring the time information of each element in an event (such as handwriting, images, documents, and audio recordings). Each element is assigned a specific timestamp, ensuring users can access event-related information. By analyzing the start and end times of elements, the system accurately categorizes elements as currently playing, already played, and yet to be played. Played and currently playing elements are rendered, and the currently playing element is played within the backtracking display area, allowing users to intuitively understand the playback status. After the currently playing element finishes playing, the system automatically controls the yet to be played element to continue playing, reducing manual intervention and improving the convenience and smoothness of operation. This data backtracking method, through independent time information management, enables users to obtain complete event information without recording, enhancing the flexibility and practicality of data backtracking. Precise element classification and dynamic feedback mechanisms allow users to easily track the playback status, improving the interactive experience and reducing operational complexity. Therefore, this provides a more flexible, comprehensive, and efficient backtracking experience, enhancing users' understanding and analysis capabilities of events.

[0026] It is worth noting that the data backtracking method of this application embodiment has broad application potential, applicable to canvas scenarios, online meetings, online education, electronic whiteboard applications, etc., effectively improving the efficiency of information acquisition and backtracking, and enhancing users' understanding and analytical abilities. In canvas scenarios, within drawing or design software, users can backtrack and view the drawing process, understanding the changes and creative journey at each step. In online meeting scenarios, within video conferencing platforms, participants can backtrack discussion content and view information on the shared screen, ensuring a comprehensive understanding of the meeting content. In online education scenarios, within online learning platforms, students can backtrack course videos, reviewing lecturers' explanations and enhancing learning effectiveness. In electronic whiteboard applications, users can backtrack previous discussions and drawing content, facilitating understanding and memorization of meeting minutes or teaching content.

[0027] Taking a canvas scene as an example, a canvas is an area used to draw graphics, text, and other visual elements. Its corresponding events can include drawing events. The canvas supports various drawing functions and can be operated via mouse, stylus, or touchscreen, providing an intuitive drawing experience. Users can draw lines, shapes, or text on the canvas. In canvas applications, the time information of each element (such as lines, shapes, and text) drawn by the user on the canvas can be recorded, ensuring that each element has a clear timestamp. Users can select the moment they wish to rewind to by sliding the timeline or clicking on a specific time point. The system will use the time information to categorize drawn elements and elements yet to be drawn based on the selected time point. During the rewind process, the drawn elements and the elements being drawn are rendered, and the element being drawn is displayed within the rewind display area. The rewind process can dynamically reproduce the user's drawing process on the canvas, gradually displaying the drawing of each element, enhancing user understanding and engagement.

[0028] Taking online meetings as an example, online meetings include corresponding meeting equipment such as cameras, microphones, large-screen displays, computers, or meeting terminals. Related video conferencing software can be installed on the computer or meeting terminal, and a meeting recording tool is also installed to record meeting content in real time. After an online meeting starts, participants join the meeting through the video conferencing software, and the host shares their screen to display the agenda or relevant documents. The meeting software has an audio recording function, and the host can start recording for later review. The meeting recording tool also transcribes participants' speeches in real time, recording text, handwriting, and key points. If an electronic whiteboard or shared document is used in the meeting, participants can write or draw on the electronic whiteboard, and the system will record the handwriting and its timestamp. After the meeting, users can choose to trigger the event review command through the interface of the meeting software or the function buttons of the meeting recording tool. For example, clicking the "View Meeting Records" or "Review Meeting Content" button. At this point, obtain the playback time information corresponding to the element in the event; determine the element currently playing based on the backtracking command; obtain the current playback time based on the playback time information corresponding to the element in the event; obtain the played element, the element currently playing, and the element to be played based on the playback time information corresponding to the element in the event and the current playback time; render the played element and the element currently playing, and make the element currently playing play within the backtracking display area; if the element currently playing finishes playing, control the elements in the element to be played to continue playing based on the playback time information corresponding to the element in the event, until all elements corresponding to the backtracking command have been responded to.

[0029] Through the above process, users can efficiently review content, and the system ensures that the order and timing of the playback content are accurate, improving the completeness and clarity of information acquisition.

[0030] In conjunction with the above embodiments, another embodiment of this application provides a data backtracking method, which can be implemented in the aforementioned computer or conference terminal. See also Figure 1 The method includes:

[0031] S11. During the event backtracking process, determine the current playback time and the element to be backtracked.

[0032] Events can include various forms of media content such as online meetings, online education, and training courses, and involve multiple interactions and information transmissions. The current playback time can be the system time corresponding to when the event was triggered for rewind. The elements to be rewound can be global elements corresponding to the event, or specific elements selected by the user within the event. These elements include handwriting (handwritten content drawn by the user on the canvas), images (illustrations or images used in the event), files (documents or materials related to the event), and recordings (audio content recorded during the event), etc.

[0033] In some embodiments, an event can be replayed using a backtracking command to determine the current playback time. A backtracking command is a command executed in the user interface that prompts the system to return to a previously recorded state or point in time to view or analyze relevant content at that point. A backtracking command may include a time parameter, i.e., the time point to be replayed; it may also include a content type, such as a drawing, video, or text; and it may include an operation type, i.e., the specific operation instructing the system to perform, such as playback, display, or rendering.

[0034] The way to trigger the rewind command can vary depending on the event scenario. For example, in a canvas drawing scenario, users can click the "Rewind" or "View Drawing History" button on the interface, or drag the timeline to select a specific point in time, and the system will automatically trigger the rewind command. In an online meeting scenario, within the meeting software, users can select the "View Meeting Records" or "Replay Meeting" option, or use preset keyboard shortcuts to quickly trigger the rewind command. In an online education scenario, users can click the "Review Course" button on the learning platform, or click a node on the course progress bar to automatically trigger rewind. In an electronic whiteboard application scenario, users can click the "Replay" or "View History" button, or perform specific gestures on the touchscreen (such as swiping left) to trigger rewind.

[0035] An element refers to any independent visual or auditory unit of content presented in a specific event. An element can be any element in the event, including handwriting, images, documents, audio recordings, etc. In multimedia events, elements are the basic units that constitute the overall content.

[0036] S12. Get the playback time information corresponding to the element in the event.

[0037] Obtaining playback time information for elements within an event can involve: A suitable data structure can be pre-configured in the system to store the time information of the event and its elements. When a user triggers a backtracking command on the interface (e.g., clicking a button or dragging the timeline), the system captures this command, retrieves all elements related to the event from the stored data, iterates through all elements in the event, and extracts the start and end times of each element. This playback time information refers to the playback time information for all elements in the event. Playback time information refers to the time data related to each element in the event (such as handwriting, images, files, recordings, etc.). It is used to determine the position and duration of each element on the timeline and can consist of start time, end time, duration, timestamp, and other information.

[0038] S13. Based on the current playback time and playback time information, classify the elements to be traced back into played elements, currently playing elements, and elements to be played.

[0039] In this process, "played elements" refers to elements whose end time is less than the current playback time. "Playing elements" refers to elements whose current playback time falls between their start and end times. "Elements to be played" refers to elements whose start time is greater than the current playback time. Therefore, each element to be backtracked can be iterated through to obtain its start and end times. The current playback time is compared with the obtained element time information, and the element is placed into the corresponding set based on the comparison result. Three sets can be pre-created: a set of played elements, a set of playing elements, and a set of elements to be played. These three sets are used to store played elements, currently playing elements, and elements to be played, respectively. Based on the playback time information, the start and end times of each element, as well as the playback order, are obtained. Elements whose start time matches the current playback time are identified as currently playing elements (this matching can be the same or close in time). Elements whose end time is less than the current playback time are identified as played elements. Elements whose start time is greater than the current playback time are identified as elements to be played.

[0040] The categorized elements can be further sorted. Specifically, based on the playback order, the currently playing elements, already played elements, and elements yet to be played can be sorted separately, resulting in sorted versions of these elements. For example, already played elements are sorted by their end time from most recent to oldest (descending order), with the most recently finished element at the top. Currently playing elements are sorted by their start time from earliest to latest (ascending order), with the elements that started playing earlier at the top. Elements yet to be played are sorted by their start time from earliest to latest (ascending order), with those about to start at the top. This sorting method makes the event rewinding process more organized and clear, allowing users to better understand and control the rewinding process.

[0041] S14. Render the played element and the element currently playing, and automatically adjust the view so that the element currently playing plays within the backtracking display area.

[0042] This feature allows rendering of both played and currently playing elements with preset effects. For example, both elements can be rendered transparent, with the played element having a higher transparency to highlight the currently playing element. It also allows for different rendering methods based on the element's category. For instance, different rendering methods can be used for handwriting and images, resulting in different rendering effects. For example, vibrant effects, animations, or gradients can be added to handwriting to increase visual appeal, while color and lighting enhancements can be added to images to emphasize their importance and dynamism.

[0043] The currently playing element can always play within the backtracking display area. The backtracking display area is used to display the currently playing element, and this area automatically adjusts based on the dynamic position and size of the playing element. For example, if the playing element goes beyond the backtracking display area, the system will automatically adjust the area's position to ensure the element remains visible. As the playing element changes, the backtracking display area should be able to track its position and state in real time to ensure the user can always see the content being played. This is achieved by automatically adjusting the view to keep the currently playing element within the backtracking display area. For example, monitoring the position (e.g., element coordinates) and size (width, height) of the playing element; determining the coordinates of the playing element relative to the backtracking display area (e.g., left, top, right, bottom boundaries); if the element's position changes (e.g., due to user interaction or the passage of time), its relationship with the backtracking display area needs to be calculated in real time; if any part of the element goes beyond the boundaries of the backtracking display area, the system will trigger a view adjustment. For example, if the element moves to the right, the backtracking area will also shift to the right to ensure the element remains within the backtracking display area. For example, the content of the backtrack display area can be reduced to ensure that the entire playing element is always fully displayed. Alternatively, the adjustment of the view extent can also be gradual. For example, as an element approaches the boundary of the backtrack area, the system can gradually adjust the position of the backtrack display area instead of moving it drastically all at once.

[0044] This adaptive mechanism allows the backtracking display area to move automatically as the position of the playing element changes, which not only improves the user experience but also ensures a smooth display of dynamic content.

[0045] S15. If the currently playing element finishes playing, control the elements in the pending element to continue playing according to the playback time information corresponding to the element in the event, and automatically adjust the view again so that the continuing playing element plays in the backtrack display area until the playback of the pending backtrack element is completed.

[0046] After the currently playing element finishes playing, the system automatically controls the playback of the next element to complete the backtracking display. During playback, each element (whether video, animation, graphics, etc.) has a playback time or duration, and the system monitors whether the currently playing element has finished playing. Within the backtracking display area, the system can maintain a queue of elements to be played. Whenever an element finishes playing, the system checks the queue to see if the next element exists. The playback order of the elements to be played is determined by their timestamps or playback times. For example, each element to be played has a predetermined playback time or timestamp, and the system determines their playback order based on the timestamps. When the current element finishes playing, the system automatically selects the next element to be played based on the order of the elements to be played and begins playback.

[0047] Additionally, whenever an element begins playing, the backtracking area automatically adjusts based on the element's current position, size, and other information. The backtracking area ensures that the playing element is always visible and maintains a smooth transition on the user interface. If the size of the playing element changes, or its position exceeds the boundaries of the current backtracking area, the view can automatically pan and scale to ensure that the element is fully displayed within the backtracking area.

[0048] Optionally, once all elements to be played have finished playing, the system can update the user interface to notify the user that the rewind process is complete. This can be done by displaying a pop-up message, a notification bar, or showing a "Rewind Complete" message on the interface.

[0049] This embodiment detects the end of the currently playing element, and the system can automatically switch to the next element to be played based on the timestamp and playback order. It also adjusts the backtracking display area in real time to ensure that the user always sees the content being played. This method can improve the continuity and smoothness of the user experience and enhance the visualization of the content.

[0050] In some embodiments of this application, see Figure 2 The above step S12, obtaining the playback time information corresponding to the element in the event, also includes:

[0051] S121. Obtain the time information corresponding to each element in the event.

[0052] The elements include handwriting, images, files, and audio recordings. The time information for each element in the event is obtained as follows: When the element is handwriting, the start and end times of the handwriting's creation during the event, as well as the relative time of each point in the handwriting relative to the start time, constitute the handwriting's time information. When the element includes images and files, the insertion times of inserting images and files during the event are obtained, constituting the image and file's time information. When the element includes audio recordings, the start and end times of recording during the event are obtained, constituting the audio recording's time information.

[0053] In the handwriting generation process, several key points are considered: time recording during the drawing process, calculation of relative time, and the impact of erasure operations on time information. The moment the user presses their finger or stylus during handwriting drawing can be considered the start time of the handwriting, and the moment the user lifts their finger or stylus can be considered the end time. Relative time records the time difference of each drawing point relative to the start time; it indicates when each point was drawn during the handwriting process. Relative time can be obtained by subtracting the start time from the actual drawing time of the point. Furthermore, when the handwriting is segmented by an erasure operation, the start and end times need to be recalculated because the erasure operation may break the integrity of the original handwriting. A new start time is calculated by combining the original start time and the first point of the relative time; a new end time is calculated by combining the original start time and the last point of the relative time. Relative time is also recalculated; for newly generated handwriting, the relative time of each point is recalculated to adapt to the new start time.

[0054] The timing information of handwriting elements ensures the integrity of the drawing process by recording the start time, end time, and the relative time of each point. When an erasure operation occurs, the regenerated timing information maintains the accuracy and consistency of the drawing. In this way, the system can effectively process and display handwriting information, whether it is drawing, erasing, or subsequent rewinding and playback.

[0055] When an element includes images and files, the insertion time is retrieved during the image insertion and file insertion events. This insertion time is the start and end time for the image and file, respectively, and their start and end times are the same by default. When an element includes audio recording, the start time of recording is the start time for the recording element, and the end time of recording is the end time for the recording element.

[0056] S122. Obtain the playback order and playback time of each element based on the time information.

[0057] This process involves organizing the time information of all elements, including handwriting, images, documents, and audio recordings. Each element's time information should include its start and end times. Using this time information, all elements are sorted by their start times. For each element, its playback time is calculated, which can be achieved by the difference between the end and start times, especially for elements like handwriting and audio recordings. For images and documents, since their start and end times are the same, the playback time can be set to a fixed value (e.g., 0.1 seconds) to allow for a brief presentation during playback. After sorting and calculating playback times, a data structure is generated containing the playback order and playback time of the elements.

[0058] S123. When the playback time interval between two adjacent elements is determined to be greater than the preset time threshold based on the playback order and playback time, the playback time interval between the two adjacent elements is adjusted to the preset time threshold.

[0059] If the playback time interval between two adjacent elements is greater than a preset time threshold (e.g., 5 seconds), the interval between them needs to be adjusted to ensure smooth playback. By adjusting the playback time interval between adjacent elements to the set time threshold, excessively long pauses during playback can be avoided.

[0060] S124. Based on the playback order, add a preset time interval between the last element and the first element. Adding a preset time interval between the last element and the first element of the entire playback sequence provides a better playback experience, ensuring a consistent time experience for the user when rewinding.

[0061] S125. Obtain the playback time interval of the two adjacent elements that have been adjusted, and the playback time of the element after adding a preset time interval between the end element and the start element. The adjusted playback time and playback order of the elements constitute the playback time information of the elements in the event.

[0062] This embodiment obtains and adjusts the playback time information of each element in the event when the backtracking command is triggered. By obtaining the time information of each element and adjusting the playback interval when necessary, the solution ensures that users can obtain a smooth and easy-to-understand content display when backtracking.

[0063] Besides optimizing playback time information by adjusting the playback interval between adjacent elements and adding a preset time interval between the last and first elements, other optimization methods can be used to ensure smoother, more precise playback and a better user experience. For example, different types of elements may require different playback intervals; video elements may need longer intervals, while text or image elements can be switched more quickly. By analyzing the type, complexity, and user needs of each element, the playback interval can be intelligently adjusted. For instance, if the system detects user interactions (such as pause, fast forward, rewind, etc.), it can adjust playback time and intervals according to user preferences. For example, when a user pauses and then resumes playback, the playback speed or interval can be automatically adjusted to provide a more tailored playback experience.

[0064] In some embodiments of this application, see Figure 3 The above steps S14, rendering the played element and the currently playing element, and automatically adjusting the view so that the currently playing element plays within the backtracking display area, also include:

[0065] S141. Render the played and currently playing elements to the preset effect.

[0066] One approach is to use a rendering layer to achieve a semi-transparent effect for both the already played and currently playing elements. Using a rendering layer to achieve this effect provides a more flexible and efficient visual presentation.

[0067] Specifically, multiple rendering layers are created. For example, a background layer renders both played and currently playing elements. Played elements have higher opacity, while currently playing elements have lower opacity to highlight the currently playing element. A current playback layer renders the currently playing element, ensuring it is opaque and on top so the user can clearly see the content. If additional visual effects (such as animation transitions, text, special effects, etc.) are needed, other layers can be set up to render these effects without affecting the main playback layer.

[0068] This step, by creating multiple rendering layers and rendering the played element and the currently playing element separately, effectively controls their transparency and hierarchy, thereby achieving flexible and efficient visual presentation.

[0069] S142. Calculate the bounding rectangle of the currently playing element and automatically adjust the size of the bounding rectangle so that the size of the bounding rectangle matches the backtracking display area, thereby allowing the currently playing element to play within the backtracking display area.

[0070] The process involves: obtaining the position and size of the currently playing element; defining an outer rectangle that encloses the element based on its position and size; scaling the outer rectangle proportionally if its width and height exceed the width and height of the backtracking display area, ensuring the adjusted rectangle fits the backtracking display area; and scaling the outer rectangle proportionally if its width and height are less than the minimum width and height of the backtracking display area, ensuring the adjusted rectangle matches the minimum width and height of the backtracking display area. If the scaled outer rectangle still exceeds the backtracking display area, it is translated to fit entirely within the backtracking display area, while preserving the visibility of surrounding elements as much as possible.

[0071] By calculating the bounding rectangle of the playing element and scaling it appropriately, the system ensures that the element is always visible within the backtracking display area, avoiding content loss or obstruction due to size mismatch, thus enhancing the flexibility and effectiveness of the application.

[0072] In some embodiments, after determining the current playback time and the element to be replayed, the method further includes: setting the element to be replayed to a preset transparent state, and adding a rendering layer on the screen corresponding to the user interface. This rendering layer is used to draw the replay effect of the element to be replayed in the aforementioned steps. The element to be replayed is set to a semi-transparent state to avoid completely obscuring the currently playing content and to ensure that the user can see both the currently playing and replaying content simultaneously. By adding an additional rendering layer on the user interface specifically for displaying the element to be replayed, the display order and visual effects of the elements can be better controlled. This rendering layer will draw the replay effect of all elements to be replayed and ensure that these elements are processed separately from the currently playing content to avoid visual clutter. This rendering layer is visually separated from other layers, located below the currently playing element layer but above the background layer. The rendering layer can draw the replay effect of the element to be replayed, such as through fade-out, blur, or color tone changes. The rendering layer is used to draw the rewind effect of the elements to be rewound in the above steps. This includes determining the current playback time and the elements to be rewound, classifying the elements to be rewound, rendering the already played and currently playing elements, automatically adjusting the view, and continuing to play the elements to be played if the currently playing element finishes playing. This achieves the content described in the above embodiments. The rendering layer is the core part of the entire rewind process. Through this layer, not only are the display effects of various elements rendered and controlled, but the rewind process is also ensured to conform to the playback order and be smoothly displayed to the user.

[0073] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0074] In conjunction with the above embodiments, another embodiment of this application provides a data backtracking device, see [link to relevant documentation]. Figure 4 The data backtracking device 200 includes:

[0075] The backtracking information acquisition module 210 is used to determine the current playback time and the element to be backtracked during the event backtracking process;

[0076] The playback time information acquisition module 220 is used to acquire the playback time information corresponding to the elements in the event;

[0077] The element classification module 230 is used to classify the elements to be traced back into played elements, currently playing elements, and elements to be played according to the current playback time and the playback time information;

[0078] The first backtracking processing module 240 is used to render the played element and the currently playing element, and automatically adjust the view so that the currently playing element plays within the backtracking display area;

[0079] The second backtracking processing module 250 is used to control the elements in the pending elements to continue playing according to the playback time information corresponding to the elements in the event if the currently playing element finishes playing, and automatically adjust the view again so that the continuing playing elements play in the backtracking display area until the playback of the pending elements is completed.

[0080] The second backtracking processing module 250 detects the end of the currently playing element and triggers the playback of the elements to be played. This module controls the playback of the elements to be played in sequence based on playback time information and backtracking logic, until all backtracked elements have finished playing. During the backtracking process, the view automatically adjusts after each element finishes playing, ensuring that the continuing elements remain within the backtracking display area and guaranteeing the smoothness and accuracy of the entire process. This dynamic playback process is based on the coordinated work of the backtracking information acquisition module 210, the playback time information acquisition module 220, the element classification module 230, and the first backtracking processing module 240, gradually completing the backtracking operation to ensure that each element plays in the correct order, state, and time, until all backtracked elements have finished playing.

[0081] The aforementioned data backtracking device 200 can be a software module, which includes several instructions stored in a memory. The processor can access the memory, call the instructions for execution, and complete the data backtracking method described in the above embodiments.

[0082] In some embodiments, the data backtracking device 200 can also be constructed from hardware devices. For example, the data backtracking device 200 can be constructed from one or more chips, and the chips can work in coordination to complete the data backtracking method described in the various embodiments. As another example, the data backtracking device 200 can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0083] It should be noted that the data backtracking device 200 described above can execute the data backtracking method for electronic devices provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the embodiments of the data backtracking device 200 can be found in the data backtracking method for electronic devices provided in the embodiments of this application.

[0084] In conjunction with the above embodiments, another embodiment of this application provides an electronic device, see [link to relevant documentation]. Figure 5 The electronic device 300 includes one or more processors 310 and a memory 320. The memory 320 is connected to one or more processors 310, for example, via a bus.

[0085] Processor 310 is configured to support the electronic device 300 in performing the corresponding functions in the data backtracking method described in the above method embodiments. Processor 310 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0086] Memory 320 is used to store program code, etc. Memory 320 may include volatile memory (VM), such as random access memory (RAM); memory 320 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 320 may also include combinations of the above types of memory.

[0087] The memory 320 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the data backtracking method in the embodiments of this application. The processor 310 executes various functional applications and data processing of the data backtracking method and data backtracking device by running the non-volatile software programs, instructions, and modules stored in the memory 320, thereby realizing the functions of each module or unit of the data backtracking method and data backtracking device provided in the above method embodiments.

[0088] The memory 320 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the data backtracking device. In some embodiments, the memory 320 may include memory remotely located relative to the processor 310, and this remote memory may be connected to the data backtracking device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0089] The one or more modules are stored in the memory 320. When executed by the one or more processors 310, they execute the data backtracking method in any of the above method embodiments. For example, they execute the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0090] The electronic device in this application embodiment may specifically be an ultra-mobile personal computer device, a smart display, or an all-in-one computer, etc.

[0091] This application provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 5 One of the processors 310 can enable the one or more processors to execute the data backtracking method in any of the above method embodiments, for example, to execute the method described above. Figure 1 Method steps S11 to S15, Figure 2 Method steps S121 to S125, Figure 3 Steps S141 to S142 in the method are implemented. Figure 4 The functions of modules 210-250 in the document.

[0092] This application provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by an electronic device, enable the electronic device to perform the data backtracking method described in any of the above method embodiments, for example, to execute the method described above. Figure 1 Method steps S11 to S15, Figure 2 Method steps S121 to S125, Figure 3 Steps S141 to S142 in the method are implemented. Figure 4 The functions of modules 210-250 in the document.

[0093] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0094] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data backtracking method, characterized in that, include: During the event rewinding process, determine the current playback time and the element to be rewinded; Obtain the playback time information corresponding to the element in the event; Based on the current playback time and the playback time information, the elements to be traced back are classified into played elements, currently playing elements, and elements to be played. Render the played element and the currently playing element, and automatically adjust the view so that the currently playing element plays within the backtracking display area; If the currently playing element finishes playing, the element in the element to be played is controlled to continue playing according to the playback time information corresponding to the element in the event, and the view is automatically adjusted again so that the continuing element plays in the backtrack display area until the playback of the element to be backtracked is completed.

2. The method according to claim 1, characterized in that, The step of obtaining the playback time information corresponding to the element in the event includes: Obtain the time information corresponding to each element in the event; The playback order and playback time of each element are obtained based on the time information. Based on the playback order and playback time of each element, the playback time is optimized to obtain optimized playback time information, which is the playback time information corresponding to the element in the event.

3. The method according to claim 2, characterized in that, The playback time is optimized based on the playback order and playback time of each element to obtain optimized playback time information. The optimized playback time information is the playback time information corresponding to the elements in the event, including: When the playback time interval between two adjacent elements is determined to be greater than a preset time threshold based on the playback order and playback time, the playback time interval between the two adjacent elements is adjusted to the preset time threshold. According to the playback order, a preset time interval is added between the last element and the first element; The playback time interval between two adjacent elements is adjusted, and the playback time of the element after adding a preset time interval between the last element and the first element is obtained. The adjusted playback time of the element and the playback order constitute the playback time information of the element in the event.

4. The method according to claim 2, characterized in that, The elements include handwriting, images, documents, and audio recordings. Obtaining the time information corresponding to each element in the event includes: When the element is handwriting, the start time and end time of the handwriting when it is generated in the event are obtained, as well as the relative time of each point of the handwriting with respect to the start time during the generation process. The start time, the end time and the relative time constitute the time information of the handwriting. When the element includes an image and a file, the insertion time corresponding to the insertion of the image and the insertion of the file in the event is obtained, and the insertion time constitutes the time information of the image and the file. When the element includes recording, the start time and end time of the recording during the event are obtained, and the start time and end time constitute the time information of the recording.

5. The method according to claim 1, characterized in that, The step of classifying the elements to be traced back into played elements, currently playing elements, and pending elements based on the current playback time and the playback time information includes: Based on the playback time information, obtain the start time, end time, and playback order of each element; The element whose start time matches the current playback time is identified as the currently playing element. Elements whose end time is less than the current playback time are identified as played elements. Elements whose start time is greater than the current playback time are identified as elements to be played. Based on the playback order of the elements, the currently playing element, the already played element, and the element to be played are sorted to obtain the sorted currently playing element, already played element, and element to be played.

6. The method according to any one of claims 1 to 5, characterized in that, The rendering of the already played element and the currently playing element, and the automatic adjustment of the view so that the currently playing element plays within the backtracking display area, includes: Render the already played element and the currently playing element to the preset effect; Calculate the bounding rectangle of the currently playing element and automatically adjust the size of the bounding rectangle so that its size matches the backtracking display area, thereby allowing the currently playing element to play within the backtracking display area.

7. The method according to claim 6, characterized in that, The step of calculating the bounding rectangle of the currently playing element and automatically adjusting the size of the bounding rectangle to match the size of the backtracking display area includes: Get the position and size of the element that is currently playing; Define an outer rectangle based on the position and size of the element being played, the outer rectangle enclosing the element being played; If the width and height of the outer rectangle are greater than the width and height of the backtracking display area, the outer rectangle is scaled proportionally so that the adjusted size of the outer rectangle fits the backtracking display area. If the width and height of the circumscribed rectangle are less than the minimum width and height of the traceback display area, then the circumscribed rectangle is enlarged proportionally so that the size of the adjusted circumscribed rectangle matches the minimum width and height of the traceback display area.

8. The method according to claim 1, characterized in that, After the step of determining the current playback time and the element to be replayed, the method further includes: The element to be backtracked is set to a preset transparent state, and a rendering layer is added to the screen corresponding to the user interface. The rendering layer is used to draw the backtracking effect of the element to be backtracked in subsequent steps.

9. An electronic device, characterized in that, include: A memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the electronic device to perform the method as described in any one of claims 1 to 8 when executing the one or more computer programs.

10. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores computer-executable instructions that, when executed by an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 8.

Images