A campus courseware visual display control method

Abstract

This application provides a method for controlling the visualization of courseware in a campus, including: collecting network bandwidth margin data and page-turning animation start and end time information to obtain potential overlap risk indicators between the current bitrate switching timing and the transition frame interval; extracting the data block occupancy required for key frame reconstruction from the adjusted switching window parameters, simulating and predicting the possibility of missing data blocks, and obtaining the predicted sharpness drop value; if the predicted sharpness drop value exceeds a preset stability threshold, dynamically extending the buffer period of the page-turning transition frame through an overlap interval identification algorithm to determine the boundary of the extended buffer period; obtaining the cross-validation result between the extended buffer period boundary and the network bandwidth margin, and determining whether to maintain a high bitrate state if the cross-validation result shows sufficient margin, otherwise reducing the bitrate according to the conventional strategy.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for controlling the visualization of courseware in a school campus. Background Technology

[0002] In modern education, the visualization and control technology of courseware is particularly crucial, directly impacting the viewing experience and knowledge transfer effectiveness of online teaching. With the rapid development of network technology, ensuring the clarity and smoothness of courseware videos under different network environments has become an important issue for improving teaching quality. This technology not only concerns students' understanding of the content but also affects the efficiency of interaction between teachers and students. However, existing methods often struggle to balance image quality and playback stability when dealing with changes in network environment and dynamic switching of courseware content. Many solutions focus more on the static allocation of network bandwidth, neglecting the impact of changes in the rhythm of content during courseware playback, especially the special needs of image transitions when teachers turn pages. This neglect leads to a disconnect between image quality adjustments and actual content changes in certain scenarios, affecting the continuity of viewing. A deeper technical challenge lies in the conflict between page-turning animations and the timing of network bitrate adjustments. Page-turning animations refer to the brief dynamic effects presented when switching pages in the courseware, while bitrate adjustment is the process of determining video clarity based on changes in network bandwidth. The overlap of these two events in time can cause problems: if the bitrate is switched at the instant the page-turning animation plays, the video decoder may fail to correctly reconstruct the image due to the loss of crucial frame data, resulting in brief blurring or blanking. The root of this conflict lies in the fact that the brief frame data from the page-turning animation is critical for image sharpness reconstruction, and the timing of the network adjustment fails to match this, causing delays or quality degradation in image restoration. Specifically, in a high-bandwidth environment, a clearer image should be achieved, but if the bitrate reduction happens to occur during page-turning animation playback, the system will lack reference frame data from the animation process and will be unable to quickly reconstruct the complete image content. For example, in a lesson, when a teacher quickly flips through pages to display multiple knowledge points, students may see a brief blur or even missing content; this discontinuity in the experience directly affects learning effectiveness. Therefore, how to coordinate the pace of page-turning animations with the timing of network bitrate adjustments during courseware playback, and avoid a decline in image quality due to the conflict between the two, becomes a key issue in improving the presentation of teaching content. Summary of the Invention

[0003] This invention provides a method for controlling the visualization of courseware in a campus, mainly including:

[0004] Obtain network bandwidth availability data and start and end time information of page-turning animation to determine the risk of overlap between bitrate switching timing and page-turning transition interval;

[0005] Based on the aforementioned overlap risk, analyze the matching degree between page-turning rhythm and bitrate switching timing, and adjust the switching window parameters accordingly;

[0006] Based on the aforementioned switching window parameters, predict the extent of the sharpness drop caused by missing keyframe data blocks;

[0007] If the sharpness drop exceeds a preset threshold, the buffer period of the page-turning transition frame is extended, and the boundary of the buffer period is determined.

[0008] Cross-validation is performed based on the buffer period boundary and the network bandwidth margin data to determine whether to maintain a high bitrate state.

[0009] If a high bitrate is maintained and data is missing, the keyframe sequence is reconstructed using reference frame data.

[0010] Based on the integrity index of the keyframe sequence, it is determined whether to perform a bitrate switch to form a screen transition path.

[0011] Furthermore, by acquiring network bandwidth availability data and the start and end times of page-turning animations, the risk of overlap between bitrate switching timing and page-turning transition intervals can be determined, including:

[0012] Periodically collect the bandwidth value and occupied traffic of the current network channel to calculate the bandwidth reserve data;

[0013] The start and end times of page-turning commands are read from the rendering end, and the pre-arranged bitrate switching trigger times are recorded to form a related dataset;

[0014] Based on the associated dataset, the bitrate switching trigger time and the page turn start and end times are compared on the time axis to determine whether they fall within the transition frame interval defined by the start time to the end time;

[0015] If it falls within the transition frame interval, it is marked as an overlapping segment, and the overlap risk is determined by the ratio of the overlapping duration to the total duration of the transition frame interval.

[0016] Furthermore, based on the aforementioned overlap risk, the matching degree between the page-turning rhythm and the timing of bitrate switching is analyzed, and the switching window parameters are adjusted, including:

[0017] Based on the aforementioned overlap risk, the animation duration corresponding to the transition frame interval is extracted from the rendering end, and the bitrate switching trigger reference time is read from the switching schedule table.

[0018] Calculate the ratio of the remaining time period from the bitrate switching trigger reference time to the end time of the transition frame interval to the duration of the animation, and compare it with a preset matching threshold to obtain the matching degree result;

[0019] If the matching degree is lower than the matching threshold, the bitrate switching trigger reference time will be postponed to after the end of the transition frame interval as the new trigger point;

[0020] If the value is not lower than the matching threshold, the bitrate switching trigger reference time is retained to form the adjusted switching window parameters.

[0021] Furthermore, regarding the switching window parameters, predicting the extent of the sharpness drop caused by missing keyframe data blocks includes:

[0022] Based on the new trigger point in the switching window parameters and the window range limited by the animation duration, video frames marked as keyframes are retrieved from the decoding buffer, and the arrival status of the data blocks required for reconstruction of each keyframe is statistically analyzed.

[0023] The arrival status of data blocks of keyframes within the window interval is randomly sampled in multiple rounds using a simulation method. Missing data blocks are marked according to the arrival probability corresponding to the network bandwidth margin data, and the missing ratio is calculated.

[0024] Based on the missing proportion, consult the pre-established sharpness drop comparison table to obtain the drop level, and take the average of the results from multiple rounds of sampling to obtain the sharpness drop magnitude.

[0025] Furthermore, if the sharpness drop exceeds a preset threshold, the buffer period of the page-turning transition frame is extended, and the buffer period boundary is determined, including:

[0026] The sharpness drop is compared with a preset stability threshold. If it exceeds the preset stability threshold, the start and end times of the transition frame interval are retrieved as the initial boundary, and the new trigger point position in the switching window parameters is obtained.

[0027] An overlapping section identification algorithm is used to sort and compare the endpoints of the transition frame interval with the interval defined by the switching window parameters to identify overlapping sections;

[0028] Using the sum of the durations of the overlapping segments as the extension length, the end time of the transition frame interval is shifted sequentially to determine the boundary of the extended buffer period.

[0029] Furthermore, cross-validation is performed based on the buffer period boundary and the network bandwidth margin data to determine whether to maintain a high bitrate state, including:

[0030] Based on the left and right boundaries of the buffer period, extract the corresponding period's margin value sequence from the network bandwidth margin data, and take the minimum value as the margin baseline value.

[0031] The baseline margin value is compared with the minimum bandwidth requirement value of the current high bitrate to obtain the cross-validation result;

[0032] If the minimum margin value is not lower than the minimum bandwidth requirement value, then the margin is determined to be sufficient, and the high bit rate state is maintained.

[0033] If the value is lower than the minimum bandwidth requirement, a lower bitrate value is obtained from the pre-established bitrate gradient table for switching.

[0034] Furthermore, if a high bitrate is maintained and data is missing, the keyframe sequence is reconstructed using reference frame data, including:

[0035] Based on the determination to maintain the high bit rate state, the keyframes entering the decoding buffer are statistically analyzed for data block occupancy, and the coordinates of the frames that have not reached the data blocks are compiled to form a missing list.

[0036] Retrieve the nearest decoded frame before the keyframe from the decoding buffer as the reference frame;

[0037] Based on the missing list, complete data blocks with the same coordinates are read from the reference frame and filled into the corresponding positions of the keyframes to form intermediate frames.

[0038] Verify the luminance and chrominance information of the intermediate frame data blocks. If the verification passes, mark them as reconstructed frames and concatenate them to form a keyframe sequence.

[0039] Furthermore, based on the integrity index of the keyframe sequence, it is determined whether to perform a bitrate switch to form a scene transition path, including:

[0040] For each data block of a reconstructed frame in the keyframe sequence, the frame-by-frame integrity ratio is calculated by comparing the number of verifications with the total number of data blocks that should arrive, and the average value is taken to obtain the integrity index.

[0041] The integrity index is compared with a preset integrity threshold. If it is higher than the integrity threshold, the bitrate is switched to the corresponding level.

[0042] If the bitrate is lower than the integrity threshold, the bitrate level before the switch is retained, and the integrity index is compared again after the end of the next page-turning transition frame interval to form the screen transition path.

[0043] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0044] This invention discloses a method for controlling the visualization of courseware in a campus environment. Addressing the conflict between page-turning animations and network bandwidth fluctuations in bitrate switching timing, the method collects information on network bandwidth availability and the start and end times of page-turning animations to analyze the risk of overlapping switching timings and transition frames. Based on preset thresholds, it adjusts switching window parameters to dynamically optimize the switching rhythm. This invention predicts the extent of sharpness drop caused by missing data blocks and extends the buffer period using an overlap interval identification algorithm to ensure smooth image transitions. Simultaneously, it maintains a high bitrate when bandwidth is sufficient or reconstructs keyframe sequences using reference frames when data is missing, ultimately achieving dual protection for bitrate switching and image integrity. The core innovation of this invention lies in resolving the conflict between page-turning animations and bitrate switching through adaptive buffering and frame reconstruction technology, significantly improving the visual smoothness and stability of courseware display and providing users with a clear and stable viewing experience. Attached Figure Description

[0045] Figure 1 This is a flowchart of a campus courseware visualization control method according to the present invention.

[0046] Figure 2 This is a schematic diagram of a campus courseware visualization control method according to the present invention.

[0047] Figure 3 This is another schematic diagram of a campus courseware visualization control method according to the present invention. Detailed Implementation

[0048] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] like Figures 1-3 This embodiment of a campus courseware visualization display control method may specifically include:

[0050] Step S101: Collect network bandwidth reserve data and page-turning animation start and end time information to obtain potential overlap risk indicators between the current bitrate switching timing and the transition frame interval.

[0051] The playback control device periodically collects the bandwidth value and occupied traffic of the current network channel, subtracting them to obtain the margin data between the bandwidth and the actual transmission demand. Simultaneously, it reads the start and end times of each page-turning command triggered from the courseware rendering end and records the bitrate switching trigger times pre-arranged by the playback control device, obtaining a correlated dataset composed of bandwidth margin, page-turning start and end times, and bitrate switching times. Based on the correlated dataset, the bitrate switching times and page-turning start and end times are compared one by one on the same timeline to determine whether the bitrate switching time falls within the transition frame interval defined by the start and end times. If it falls within the transition frame interval, the time period is marked as an overlapping segment, and the potential overlap risk index is obtained by using the ratio of the overlapping duration to the total duration of the transition frame interval.

[0052] In one embodiment, the playback control device is deployed between the courseware live streaming server and the student viewing terminal, undertaking the dual tasks of network channel monitoring and bitrate scheduling.

[0053] Specifically, the bandwidth value is obtained by periodically sending query commands to the network interface card, typically once per second. The occupied traffic is obtained by counting the cumulative number of bytes of video data packets pushed to student viewing terminals at the current moment. The bandwidth reserve data is obtained by subtracting the occupied traffic from the bandwidth value, reflecting the additional data volume that the current network channel can carry on top of ensuring existing transmission.

[0054] It should be noted that the page-turning command is triggered by the courseware rendering end on the teacher's end the instant the teacher clicks the page-turning button. The courseware rendering end takes the triggering time of the page-turning command as the start time and the end time of the page-turning transition animation as the end time. The time interval between the two is the transition frame interval.

[0055] Preferably, the transition frame interval carries the gradual image data of the courseware page switching, which is of reference value for image clarity reconstruction. The bitrate switching trigger time is pre-scheduled by the playback control device based on the changing trend of bandwidth margin data and recorded in the switching plan table.

[0056] Specifically, the timeline comparison process positions the bitrate switching trigger time and each set of start and end times on the same millisecond-level timeline, and determines whether the bitrate switching trigger time is within the transition frame interval defined by the start and end times. If it falls within the transition frame interval, the time period between the bitrate switching trigger time and the end time is extracted as the overlapping segment. The duration of this overlapping segment is divided by the total duration of the transition frame interval, and the resulting ratio is the potential overlap risk index. The larger the ratio, the deeper the conflict between the bitrate switching and page turning transition.

[0057] Step S102: Based on the overlap risk index, analyze the matching degree between the animation duration under page-turning rhythm control and the bitrate switching trigger benchmark, and determine if the matching degree is lower than the threshold to delay the switching timing, thereby obtaining the adjusted switching window parameters.

[0058] Based on the potential overlap risk index, the animation duration corresponding to the transition frame interval is extracted from the page-turning rhythm control record of the courseware rendering end, and the bitrate switching trigger reference time is read from the switching plan table of the playback control device. The remaining time period between the bitrate switching trigger reference time and the end time of the transition frame interval is divided by the animation duration, and the resulting ratio is compared with a preset matching threshold to obtain a matching degree judgment result reflecting the rhythm coordination degree. According to the matching degree judgment result, if the matching degree is lower than the matching threshold, the bitrate switching trigger reference time is postponed to after the end time of the transition frame interval, and the postponed time is used as the new trigger point after the delay; if the matching degree is not lower than the matching threshold, the original bitrate switching trigger reference time is retained as the new trigger point; the new trigger point and the animation duration are combined to form the adjusted switching window parameters.

[0059] The output of the potential overlap risk index can only reflect whether the bitrate switching time falls within the transition frame interval, but it cannot directly characterize the degree of coordination between the page-turning animation rhythm and the bitrate switching reference time in terms of time allocation.

[0060] Specifically, when a page-turning command is triggered, the courseware rendering terminal controls and records the gradual playback rhythm of the animation according to the pre-set page-turning rhythm. The duration of the animation is the time span from the start time to the end time in the page-turning rhythm control record, which is usually expressed as the total time consumed by transition actions such as page fading, panning, or scaling.

[0061] In one embodiment, the matching degree is obtained as follows: The bitrate switching trigger reference time is denoted as T1, the end time of the transition frame interval is denoted as T2, and the animation duration is denoted as D. The remaining time period is T2 minus T1, and the matching degree M equals (T2 − T1) divided by D. The matching degree M reflects the proportion of time that can still be occupied by the animation process between the bitrate switching trigger reference time and the completion of the page-turning transition. The smaller the proportion, the closer the bitrate switching reference time is to the end of the page-turning transition or even to the middle of the transition, the more limited the margin for the complete presentation of the animation, and the more severe the rhythm conflict.

[0062] It should be noted that the value of the matching threshold is determined empirically based on the minimum recognizable duration of the gradual animation in the courseware screen.

[0063] In one possible implementation, the value can be a small number between 0.3 and 0.5. When the matching degree M is lower than this matching threshold, it means that the bitrate switching trigger reference time has encroached on the time share within the page-turning transition frame interval that should have been reserved for the gradient screen.

[0064] Specifically, in scenarios where teachers rapidly and continuously flip through pages to explain multiple knowledge points, the animation duration recorded by the courseware rendering end is often short, and the matching degree is easily lower than the matching threshold, thus entering a delayed process. Further, in low-matching-degree scenarios, the playback control device executes the delayed action as follows: the bitrate switching trigger reference time T1 is directly reset to the transition frame interval end time T2 plus a buffer offset Δt. The buffer offset Δt is determined by the minimum interval between two adjacent page-turning instructions recorded by the page-turning rhythm control of the courseware rendering end, thus ensuring the new trigger point is located after the transition animation has completely ended, avoiding bitrate switching before the gradual image has stabilized. The new trigger point, along with the original animation duration D, is written into the switching window parameters. The switching window parameters contain two components: the absolute time of the new trigger point and the window width represented by the animation duration D. These two components together define the legal time interval for bitrate switching execution.

[0065] Understandably, for scenarios where the matching degree is not lower than the matching threshold, the bitrate switching trigger reference time is originally located outside the transition frame interval or occupies only a very small tail portion, and the playback control device directly uses the original trigger reference time as the new trigger point. Regardless of whether it has been delayed, the output switching window parameters are presented as a tuple of the new trigger point and the animation duration D, for use by the downstream bitrate switching execution stage.

[0066] Step S103: Extract the data block occupancy information required for keyframe reconstruction from the adjusted switching window parameters, simulate and predict the possibility of missing data blocks, and obtain the predicted sharpness drop value.

[0067] Based on the new trigger point in the adjusted switching window parameters and the window range defined by the animation duration, video frames marked as keyframes within this window range are retrieved from the decoding buffer of the playback control device. For each keyframe, the number of successfully arrived data blocks and the total number of data blocks that should arrive are counted among all the data blocks upon which its reconstruction depends, resulting in a record of the occupancy of data blocks required for keyframe reconstruction. Based on this occupancy record, a Monte Carlo simulation method is used to perform multiple rounds of random sampling on the arrival status of data blocks for each keyframe within the window range. In each round of sampling, some data blocks are randomly marked as missing according to the arrival probability of data blocks corresponding to the network bandwidth margin data. The proportion of missing data blocks to the total number of data blocks that should arrive is counted, and this proportion is looked up in a pre-established sharpness drop comparison table to obtain the corresponding sharpness level drop level. The average of the drop levels obtained from multiple rounds of sampling is taken to obtain the predicted sharpness drop magnitude value.

[0068] Based on the adjusted switching window parameters, the window interval jointly defined by the new trigger point and the animation duration contains keyframes used to connect the image clarity before and after the bitrate switch. These keyframes are image reference points in the video encoding system that can be decoded independently without relying on other frames; their integrity directly determines the speed at which image clarity recovers during the bitrate switch.

[0069] Specifically, the keyframes are divided into multiple data blocks arranged according to screen coordinates during encoding. Each data block corresponds to a small rectangular area on the screen, carrying the brightness and chromaticity information of that area. The occupancy record is obtained by the playback control device scanning frame by frame within the window interval. The record lists the total number of data blocks that should be reached, the number of data blocks that have been successfully reached, and the position of the data blocks that have not been reached in the screen coordinate grid for each keyframe.

[0070] In one embodiment, for a single frame of courseware, the total number of data blocks to be reached is determined based on the screen resolution and the size of each block, while the number of data blocks already reached is determined by counting the sequence number range of received messages in the decoding buffer. The occupancy record organizes these two types of data into a frame-by-frame ledger by frame index, serving as the input for subsequent simulation and prediction.

[0071] It should be noted that the Monte Carlo simulation method is a statistical method that approximates the true probability distribution by using a large number of random samples. Here, the Monte Carlo simulation method is used to estimate the actual degree of missing keyframes under the current network conditions.

[0072] In one embodiment, the Monte Carlo simulation method is executed as follows: Using the arrival probability p of the data block corresponding to the network bandwidth margin data as the probability of a single block successfully arriving, for each keyframe in the occupancy record, the total number of data blocks that should arrive is recorded as N. Each data block is independently determined to arrive according to probability p; those that do not arrive are marked as missing. After one round of sampling, the number m of missing data blocks in that frame is counted, and the missing ratio r is calculated as m divided by N. Multiple rounds of sampling are repeated in the same manner, and the missing ratio r obtained from each round of sampling is independently retained. The arrival probability p of the data block is obtained by dividing the network bandwidth margin data by the transmission rate required by the current bitrate and taking a truncation value less than or equal to 1. The more abundant the margin, the closer p is to 1. Further, the sharpness drop-off reference table is a pre-established lookup table. Each row in the table corresponds to a missing ratio range, and the right side of each row records the decrease in image sharpness level after decoding under that missing ratio range. The decrease in level uses an integer scale; the larger the scale, the deeper the degree of blurring or blockiness in the image. The missing proportion r obtained from each round of sampling is used to find the corresponding row's decrease level in the sharpness fallback comparison table.

[0073] It is understandable that the arithmetic mean of the decline levels obtained from multiple rounds of sampling is the predicted sharpness drop magnitude. This magnitude reflects the statistically expected level of the possible decline in image sharpness when bitrate switching is performed within the time range defined by the switching window parameter.

[0074] Step S104: If the predicted sharpness drop exceeds the preset stability threshold, the buffer period of the page-turning transition frame is dynamically extended using the overlap interval identification algorithm to determine the boundary of the extended buffer period.

[0075] The predicted sharpness drop magnitude is compared with a preset stability threshold. If the drop magnitude exceeds the stability threshold, the start and end times of the transition frame interval are retrieved as initial boundaries. The location identifier of the new trigger point is extracted from the switching window parameters to obtain the initial boundary dataset for carrying out the expansion operation. Based on the initial boundary dataset, an overlapping interval identification algorithm is used to sort the endpoints of the transition frame interval and the window interval defined by the switching window parameters on the same time axis and compare them point by point to identify the overlapping segments and output the start and end times of each overlapping segment. The overlapping interval identification algorithm refers to a method of merging and scanning the endpoints of two sets of interval sequences with start and end times to obtain a set of overlapping segments. Based on the sum of the durations of each overlapping segment in the set of overlapping segments as the extension length, the end time of the transition frame interval is shifted backward according to the extension length. The start time of the transition frame interval is retained as the left boundary, and the time obtained after shifting is used as the right boundary to determine the boundary of the expanded buffer period.

[0076] The predicted sharpness drop value reflects the statistically expected decrease in image sharpness when the bitrate switching is performed within the range defined by the switching window parameters. The stability threshold is a pre-established upper limit of the acceptable decrease in image sharpness, determined by the playback control device based on the visual tolerance configuration of the courseware's gradual animation.

[0077] In one possible implementation, the stability threshold is a value between an integer scale of 1 and 2, with a smaller scale indicating a higher requirement for image stability.

[0078] Specifically, when the sharpness drop exceeds the stability threshold, it means that even if the new trigger point in the switching window parameters remains unchanged, the image will still exhibit blurring or blockiness noticeable to the viewer during bitrate switching. In this scenario, the end of the transition frame interval needs to be further extended to allow the timing of bitrate switching to be within a wider buffer range.

[0079] It should be noted that the initial boundary dataset consists of three parts: the start time T0 and end time Tend of the transition frame interval, and the time Tn corresponding to the new trigger point in the switching window parameters. The initial boundary dataset serves as input for subsequent interval expansion operations, carrying two types of information: the time span of the transition animation playback and the time position of the bitrate switching trigger.

[0080] It is understood that the overlapping interval identification algorithm is a method for processing the relationship between one-dimensional interval sets. Its core lies in determining the overlapping segments of two sets of intervals by merging endpoints.

[0081] Specifically, the execution process of the overlapping interval identification algorithm is as follows: The transition frame interval is represented as a first interval starting from T0 and ending at Tend; the window interval defined by the switching window parameter is represented as a second interval starting from Tn and ending at the time obtained by adding the animation duration D to Tn; the four endpoints of the first and second intervals are sequentially sorted according to time to obtain four ordered time points; during scanning, an interval state counter is maintained, and the counter is incremented or decremented by one each time an endpoint is passed. When the counter value changes from 1 to 2, the time is recorded as the starting endpoint of a certain overlapping segment; when the counter value changes from 2 to 1, the time is recorded as the ending endpoint of the overlapping segment, thus obtaining a set of overlapping segments composed of paired starting and ending endpoints. Each segment in the set of overlapping segments reflects the intersection of the transition animation playback period and the bitrate switching execution period on the time axis. Furthermore, for scenarios where multiple adjacent page-turning commands are triggered consecutively, the set of overlapping segments may contain multiple segments.

[0082] In one embodiment, the duration of each overlapping segment in the overlapping segment set is sequentially summed to obtain an extension length L. The extension length L is obtained by subtracting the start endpoint from the end endpoint of each overlapping segment, and then summing the durations of all overlapping segments. The end time Tend of the transition frame interval is shifted forward according to the extension length L, and the resulting new end time Tnew is equal to Tend plus L. The shifting process only applies to the tail endpoint of the transition frame interval, while the front endpoint remains unchanged.

[0083] Preferably, the extended buffer period uses the original start time T0 of the transition frame interval as its left boundary and the new end time Tnew obtained after the shift as its right boundary. The left and right boundaries together constitute the boundary of the extended buffer period. The boundary of the extended buffer period covers the original transition animation playback segment and all overlapping segments that may be triggered by bitrate switching.

[0084] For example, in a scenario where a teacher quickly flips through pages to explain multiple knowledge points, the transition animation between two adjacent page flips and the bitrate switching trigger time overlaps significantly. The cumulative duration L of the overlapping segment set is often significantly greater than zero, and the boundary of the extended buffer period shifts backward accordingly, providing an extended time window for the stable recovery of image clarity.

[0085] Step S105: Obtain the cross-validation result of the extended buffer period boundary and the network bandwidth margin. If the cross-validation result shows that the margin is sufficient, maintain the high bitrate state; otherwise, reduce the bitrate according to the conventional strategy.

[0086] Based on the left and right boundaries defined by the extended buffer period boundary, the sequence of remaining values ​​at each sampling moment between the left and right boundaries is extracted from the network bandwidth remaining data, and the minimum value is taken as the minimum remaining value for the buffer period. The minimum remaining value is compared numerically with the minimum bandwidth requirement corresponding to the current high bitrate level of the playback control device to obtain a cross-validation result representing the boundary and remaining value matching relationship. According to the cross-validation result, if the minimum remaining value is not lower than the minimum bandwidth requirement value, it is determined that the remaining value is sufficient, and the playback control device continues to push video data to the decoding buffer at the original high bitrate level; if the minimum remaining value is lower than the minimum bandwidth requirement value, the bitrate value of the next lower level is retrieved from the pre-established bitrate gradient table, which stores the bitrate values ​​of each level in descending order of resolution. The playback control device switches to the lower bitrate level to push video data to the decoding buffer, thus obtaining the bitrate level selection result.

[0087] Following the extended buffer period boundary, the left boundary is the start time of the transition frame interval, and the right boundary is the new end time obtained after the shift. Together, they define a time window for restoring image clarity. The cross-validation involves matching the network bandwidth margin data within this time window with the resource level required to maintain high bitrate transmission, in order to determine whether the physical conditions are met to continue carrying high bitrate video streams during the buffer period.

[0088] Specifically, the network bandwidth margin data is obtained by the playback control device periodically querying the network interface card, with each sampling moment corresponding to a margin value.

[0089] In one embodiment, the process of extracting the surplus value sequence is as follows: starting from the left boundary and ending at the right boundary, all sampling points falling within the time period are extracted from the time axis of the surplus data and arranged in chronological order to obtain the surplus value sequence; each surplus value in the sequence is scanned, and the smallest value is recorded as the minimum surplus value. The minimum surplus value represents the surplus level that can be provided at the moment when bandwidth resources are most strained during the entire buffer period. This minimum value extraction method ensures that subsequent judgments are based on the most unfavorable resource situation, rather than the average level.

[0090] It should be noted that the minimum bandwidth requirement is obtained by the playback control device based on the current high bitrate setting's bitrate parameter plus a certain protection margin. The bitrate parameter is the number of bytes per second of video data stream. Furthermore, for scenarios where the margin threshold is not lower than the minimum bandwidth requirement, the playback control device does not initiate any bitrate switching action; the existing high bitrate setting continues to operate in the decoding buffer, and the video data stream is continuously pushed to the student's viewing terminal according to the current resolution level. This scenario is common in classroom environments where network bandwidth is ample and the teacher's page-turning pace is slow.

[0091] It is understood that the bitrate gradient table is a lookup table pre-established in the playback control device. Each row in the table is arranged in descending order of resolution level, and each row records a bitrate value and the corresponding resolution level identifier.

[0092] In one possible implementation, the bitrate gradient table is arranged from high to low with four levels: Ultra HD, High Definition, Standard Definition, and Smooth. There is a certain difference between the bitrate values ​​of each adjacent level. When the margin threshold is lower than the minimum bandwidth requirement, the playback control device locates the current level in the bitrate gradient table, looks up the bitrate value of the next adjacent level, and switches the target output bitrate of the decoding buffer to that adjacent level. The video data stream continues to be pushed according to the downgraded resolution level.

[0093] Preferably, the bitrate selection result is recorded in the operation log of the playback control device in the form of a binary tuple, which includes the bitrate identifier and the corresponding bitrate value used in this selection.

[0094] For example, in a scenario where a teacher rapidly flips through pages to explain multiple knowledge points, causing instantaneous network fluctuations, the margin baseline value is often lower than the minimum bandwidth requirement value. The playback control device then switches down one level according to the bitrate gradient table to avoid data transmission overflow during the buffer period.

[0095] Step S106: When it is necessary to maintain a high bit rate and there is missing image data, take out the complete reference frame data block from the decoded previous frame, and fill the missing data block with the same coordinates in the reference frame into the missing position according to the coordinate position of the missing data block in the image, so as to form the reconstructed keyframe sequence.

[0096] Based on the determination to maintain a high bitrate in the bitrate selection result, the occupancy record of the data blocks required for keyframe reconstruction is retrieved for the keyframe currently entering the decoding buffer. Data blocks marked as unreached are organized into a missing list according to their horizontal and vertical coordinate positions in the image coordinate grid, resulting in a keyframe to be repaired carrying the missing block location information. Based on the keyframe to be repaired and its missing list, the most recent frame that has been decoded before the keyframe to be repaired is retrieved from the decoding buffer as a reference frame. For each set of horizontal and vertical coordinate positions in the missing list, complete image data blocks at the same horizontal and vertical coordinate positions are read from the reference frame and written into the corresponding empty positions of the keyframe to be repaired according to their original coordinate positions, resulting in an intermediate frame after block filling. The intermediate frames are checked block by block to verify the completeness of the brightness and chromaticity information of all image data blocks. Specifically, the mean square error (MSE) of the pixel brightness Y component and chromaticity Cb and Cr components is calculated for each data block. The formula is MSE = (1 / N)∑(original value - current value)², where N is the number of pixels. If the MSE of all blocks is less than the threshold of 0.5, it is considered to have passed the verification and the intermediate frame is marked as a reconstructed frame. The reconstructed frames are then concatenated in sequence according to the time sequence number of the keyframes in the decoding buffer to obtain the reconstructed keyframe sequence.

[0097] Following the bitrate selection result, when the playback control device determines that a high bitrate level should be maintained, the key frames entering the decoding buffer still carry picture information at the ultra-high definition or high definition level. Each key frame is divided into multiple picture data blocks arranged according to the picture coordinate grid during encoding. Each picture data block corresponds to a small rectangular area on the picture and carries the brightness and chromaticity information of that area.

[0098] Specifically, in scenarios where there is momentary packet loss or latency jitter in the network channel, some image data blocks, although belonging to the data that should arrive in the keyframe, may have partial gaps during decoding because the packets do not arrive in the decoding buffer within the decoding time limit. This phenomenon is common in live courseware presentations when teachers quickly flip through pages, and the characters or illustrations on the screen may appear blocky or have blurred edges.

[0099] It should be noted that the process of compiling the missing data list is as follows: The playback control device retrieves the occupancy records of the data blocks required for the reconstruction of the keyframe. For data blocks marked as unreached, its corresponding x-coordinate value and y-coordinate value are read sequentially. These two coordinate values ​​are combined into a binary position identifier (x, y), and then compiled into a position list by frame index. Each record in the position list corresponds to a position identifier of a missing frame on the corresponding keyframe.

[0100] It is understood that the keyframe to be repaired is the current keyframe itself carrying the missing list, and the image data blocks that should have been there have been partially in place and partially missing.

[0101] Specifically, the selection of the reference frame is based on the principle of inter-frame image correlation: in live courseware broadcasting, the image changes between adjacent frames are usually limited to the gradual transition area of ​​page turning or the local movement of the explanation cursor, and most areas of the image remain stable between adjacent frames. The selection process of the reference frame is as follows: the playback control device backtracks in the decoding buffer according to the playback time sequence, locates the image of the most recent frame that has been fully decoded before the key frame to be repaired, and uses the entire image of that frame as the reference frame.

[0102] In one embodiment, the coordinate position matching filling process is as follows: The playback control device traverses each binary position identifier (x, y) in the missing list, and for each identifier, reads the corresponding image data block from the reference frame using the (x, y) coordinates. The reading process extracts an image data block from the reference frame corresponding to the missing position on the keyframe to be repaired, based on the same coordinates in the image coordinate grid. The extracted content includes the luminance and chrominance information carried by the data block. The extracted image data block is written back to the corresponding missing position on the keyframe to be repaired according to its original (x, y) coordinate values, changing the position from a missing state to a state carrying image data. The traversal process performs a read and write operation once for all position identifiers in the missing list. After all position identifiers are processed, all missing positions on the keyframe to be repaired are covered by image data blocks with the same coordinates from the reference frame, resulting in a filled intermediate frame.

[0103] It should be noted that although the intermediate frame has a complete number of image data blocks, some data blocks come from the reference frame rather than the actual image of the current frame, and their integrity needs to be further verified.

[0104] Preferably, the verification process for the luminance and chrominance information is as follows: The playback control device scans the intermediate frames block by block, reads the effective byte count of the luminance channel and the effective byte count of the chrominance channel for each frame data block, and compares them with the byte count that the data block should have under the encoding format; if the effective byte count of each frame data block is consistent with the required byte count, then the data block is determined to have passed the verification; when all frame data blocks of the intermediate frame have passed the verification, the intermediate frame is marked as a reconstructed frame. This verification process simultaneously covers the originally arrived data blocks and the data blocks filled in from the reference frame.

[0105] It is understood that the decoding buffer contains multiple keyframes according to their playback time sequence, and each keyframe is processed independently according to the aforementioned filling and verification process. The playback control device concatenates the reconstructed frames sequentially according to the time sequence of the keyframes, with the reconstructed frames of the earlier sequence number placed before the reconstructed frames of the later sequence number placed after the reconstructed frames of the later sequence number. The concatenated multi-frame sequence is the reconstructed keyframe sequence, which serves as the source of the image output downstream of the decoding buffer.

[0106] In step S107, the reconstructed keyframe sequence is sent to the decoding of the playback control device. If the sequence integrity is higher than the threshold, the bitrate is switched. If the integrity is lower than the threshold, the previous bitrate is used for playback until the next page-turning animation ends and the switching is attempted again, forming a clear and smooth transition path for the picture.

[0107] Based on the reconstructed keyframe sequence, the number of data blocks that pass verification in each reconstructed frame is divided by the total number of data blocks that should arrive in that frame to obtain a frame-by-frame integrity ratio. Then, the arithmetic mean of the integrity ratios of all reconstructed frames in the keyframe sequence is taken to obtain the sequence integrity index of the keyframe sequence. This sequence integrity index is then sent to the downstream decoding module of the decoding buffer of the playback control device. The sequence integrity index is compared with a preset integrity threshold. If the sequence integrity index is higher than the integrity threshold, the playback control device executes the bitrate switch in the bitrate selection result to the corresponding level and continuously inputs video data to the decoding module at that level. If the sequence integrity index is lower than the integrity threshold, the bitrate level before the switch is retained to continue playback. The sequence integrity index is compared again after the end of the transition frame interval corresponding to the next page-turning instruction to obtain a smooth transition path for a clear picture.

[0108] Following the reconstructed keyframe sequence, the sequence integrity index reflects the overall decodeability of the keyframe sequence when it is sent to the downstream decoding module of the decoding buffer of the playback control device. The decoding module is an execution unit in the video playback system responsible for restoring the compressed bitstream to pixel images. It is highly sensitive to the integrity of the input frame sequence; the higher the proportion of missing data blocks, the greater the probability of mosaic or frozen frames in the output image.

[0109] Specifically, the frame-by-frame integrity ratio is obtained as follows: For each reconstructed frame in the keyframe sequence, the number of image data blocks that pass the luminance and chrominance information verification is denoted as n, and the total number of image data blocks that the frame should reach is denoted as N. The fractional value of n divided by N is the integrity ratio of the reconstructed frame. The arithmetic mean is obtained by summing the integrity ratios of all reconstructed frames in the keyframe sequence, then dividing by the total number of reconstructed frames. The average value is used as the sequence integrity index. This index ranges from 0 to 1; the closer the value is to 1, the higher the overall decodeability of the sequence.

[0110] It should be noted that the integrity threshold is preset by the playback control device based on the minimum load requirement of the decoding module for the input frame.

[0111] In one possible implementation, a small value between 0.85 and 0.95 is taken.

[0112] It is understandable that, for scenarios where the sequence integrity index is higher than the integrity threshold, the playback control device determines that the currently reconstructed keyframe sequence is sufficient to carry the output of the picture after the bitrate switch. According to the corresponding level identifier and bitrate value recorded in the bitrate level selection result, the target input bitrate of the decoding module is switched to the level. The decoding module parses the input bitstream according to the bitrate parameters of the new level and pushes the clear picture after the switch to the student viewing terminal.

[0113] Specifically, in scenarios where the sequence integrity index is lower than the integrity threshold, the playback control device does not initiate the level switching action. The decoding module continues to receive and parse video data at the bitrate level before the switch, and the output image maintains the original clarity level. Simultaneously, the playback control device enters a waiting state, listening for the next page-turning command from the courseware rendering end. When the end time of the transition frame interval corresponding to the next page-turning command arrives, the device retrieves the image data block verification result of the reconstructed frame within that transition frame interval, updates the sequence integrity index in the same way, and compares the updated sequence integrity index with the integrity threshold again. If it is still lower than the threshold, the device retains the bitrate level before the switch and waits for the end time of the next transition frame interval until the comparison result is higher than the threshold before performing level switching.

[0114] Preferably, the smooth transition path of the clear picture is formed by connecting the picture output frame before the bitrate level switch, the gradual picture in the transition frame interval, and the picture output frame after the bitrate level switch in sequence according to the playback time. There are no picture gaps or sharpness jumps between the frames on the path.

[0115] For example, in a scenario where a teacher rapidly flips through pages to explain multiple knowledge points, the playback control device may span multiple transition frame intervals before completing a single gear switch. The smooth transition path of the clear picture is extended accordingly to cover multiple page-flipping transitions, and the improvement in picture clarity is embedded in the natural page-flipping rhythm.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling the visual display of courseware in a campus, characterized in that, include: Obtain network bandwidth availability data and start and end time information of page-turning animation to determine the risk of overlap between bitrate switching timing and page-turning transition interval; Based on the aforementioned overlap risk, analyze the matching degree between page-turning rhythm and bitrate switching timing, and adjust the switching window parameters accordingly; Based on the aforementioned switching window parameters, predict the extent of the sharpness drop caused by missing keyframe data blocks; If the sharpness drop exceeds a preset threshold, the buffer period of the page-turning transition frame is extended, and the boundary of the buffer period is determined. Cross-validation is performed based on the buffer period boundary and the network bandwidth margin data to determine whether to maintain a high bitrate state. If a high bitrate is maintained and data is missing, the keyframe sequence is reconstructed using reference frame data. Based on the integrity index of the keyframe sequence, it is determined whether to perform a bitrate switch to form a screen transition path.

2. The campus courseware visualization display control method as described in claim 1, characterized in that, The process of acquiring network bandwidth reserve data and the start and end times of page-turning animations to determine the risk of overlap between bitrate switching timing and page-turning transition intervals includes: Periodically collect the bandwidth value and occupied traffic of the current network channel to calculate the bandwidth reserve data; The start and end times of page-turning commands are read from the rendering end, and the pre-arranged bitrate switching trigger times are recorded to form a related dataset; Based on the associated dataset, the bitrate switching trigger time and the page turn start and end times are compared on the time axis to determine whether they fall within the transition frame interval defined by the start time to the end time; If it falls within the transition frame interval, it is marked as an overlapping segment, and the overlap risk is determined by the ratio of the overlapping duration to the total duration of the transition frame interval.

3. The campus courseware visualization display control method as described in claim 1, characterized in that, The step of analyzing the matching degree between page-turning rhythm and bitrate switching timing based on the overlap risk, and adjusting the switching window parameters, includes: Based on the aforementioned overlap risk, the animation duration corresponding to the transition frame interval is extracted from the rendering end, and the bitrate switching trigger reference time is read from the switching schedule table. Calculate the ratio of the remaining time period from the bitrate switching trigger reference time to the end time of the transition frame interval to the duration of the animation, and compare it with a preset matching threshold to obtain the matching degree result; If the matching degree is lower than the matching threshold, the bitrate switching trigger reference time will be postponed to after the end of the transition frame interval as the new trigger point; If the value is not lower than the matching threshold, the bitrate switching trigger reference time is retained to form the adjusted switching window parameters.

4. The campus courseware visualization display control method as described in claim 1, characterized in that, The prediction of the sharpness drop caused by missing keyframe data blocks for the switching window parameters includes: Based on the new trigger point in the switching window parameters and the window range limited by the animation duration, video frames marked as keyframes are retrieved from the decoding buffer, and the arrival status of the data blocks required for reconstruction of each keyframe is statistically analyzed. The arrival status of data blocks of keyframes within the window interval is randomly sampled in multiple rounds using a simulation method. Missing data blocks are marked according to the arrival probability corresponding to the network bandwidth margin data, and the missing ratio is calculated. Based on the missing proportion, consult the pre-established sharpness drop comparison table to obtain the drop level, and take the average of the results from multiple rounds of sampling to obtain the sharpness drop magnitude.

5. The campus courseware visualization display control method as described in claim 3, characterized in that, If the sharpness drop exceeds a preset threshold, the buffer period of the page-turning transition frame is extended, and the buffer period boundary is determined, including: The sharpness drop is compared with a preset stability threshold. If it exceeds the preset stability threshold, the start and end times of the transition frame interval are retrieved as the initial boundary, and the new trigger point position in the switching window parameters is obtained. An overlapping section identification algorithm is used to sort and compare the endpoints of the transition frame interval with the interval defined by the switching window parameters to identify overlapping sections; Using the sum of the durations of the overlapping segments as the extension length, the end time of the transition frame interval is shifted sequentially to determine the boundary of the extended buffer period.

6. The campus courseware visualization display control method as described in claim 1, characterized in that, The step of cross-validating based on the buffer period boundary and the network bandwidth margin data to determine whether to maintain a high bitrate state includes: Based on the left and right boundaries of the buffer period, extract the corresponding period's margin value sequence from the network bandwidth margin data, and take the minimum value as the margin baseline value. The baseline margin value is compared with the minimum bandwidth requirement value of the current high bitrate to obtain the cross-validation result; If the minimum margin value is not lower than the minimum bandwidth requirement value, then the margin is determined to be sufficient, and the high bit rate state is maintained. If the value is lower than the minimum bandwidth requirement, a lower bitrate value is obtained from the pre-established bitrate gradient table for switching.

7. The campus courseware visualization display control method as described in claim 1, characterized in that, If a high bitrate is maintained and data is missing, the keyframe sequence is reconstructed using reference frame data, including: Based on the determination to maintain the high bit rate state, the keyframes entering the decoding buffer are statistically analyzed for data block occupancy, and the coordinates of the frames that have not reached the data blocks are compiled to form a missing list. Retrieve the nearest decoded frame before the keyframe from the decoding buffer as the reference frame; Based on the missing list, complete data blocks with the same coordinates are read from the reference frame and filled into the corresponding positions of the keyframes to form intermediate frames. Verify the luminance and chrominance information of the intermediate frame data blocks. If the verification passes, mark them as reconstructed frames and concatenate them to form a keyframe sequence.

8. The campus courseware visualization display control method as described in claim 1, characterized in that, The step of determining whether to perform a bitrate switch and form a screen transition path based on the integrity index of the keyframe sequence includes: For each data block of a reconstructed frame in the keyframe sequence, the frame-by-frame integrity ratio is calculated by comparing the number of verifications with the total number of data blocks that should arrive, and the average value is taken to obtain the integrity index. The integrity index is compared with a preset integrity threshold. If it is higher than the integrity threshold, the bitrate is switched to the corresponding level. If the bitrate is lower than the integrity threshold, the bitrate level before the switch is retained, and the integrity index is compared again after the end of the next page-turning transition frame interval to form the screen transition path.

Images