Intelligent processing method and system for training video

Abstract

The application provides a kind of intelligent processing method and system of training video, belong to video processing technical field, the application is first to training video frame segmentation and content identification, divide out the key area of bearing teaching content, the edge area formed by key area extension and the non-key area adjacent thereto, then generate two encoding reconstruction versions with different fidelity levels based on three types of areas, collect student response item data such as pause, playback, answer error after playing and compare, determine the area with different fidelity levels between the two versions as the quality related area, and then update the encoding protection area range of the corresponding knowledge point, and adjust the encoding parameters and reconstruction parameters of the area in the range in the unplayed video frame and review video segment. The application can identify the specific area that the quality change has actual impact on learning effect by comparing the student playback feedback data corresponding to different fidelity level encoding versions.

Description

Technical Field

[0001] This application belongs to the field of video processing technology, and in particular relates to a method and system for intelligent processing of training videos. Background Technology

[0002] With the advancement of digital transformation in the education and training industry, pre-recorded courses and online training have become the mainstream teaching methods. Training videos include a large amount of teaching content such as blackboard writing, formulas, and practical demonstrations. The video processing effect directly affects students' learning experience and has a wide range of application needs in K-12 education, vocational skills training, and adult education.

[0003] Currently, training video processing mainly uses a unified encoding and compression method to compress and transmit the video, applying the same or similar encoding parameters to different areas of the screen. Some solutions, based on this, combine students' pause frequency, replay frequency and other behavioral data to analyze teaching difficulties, in order to help teachers understand the students' learning situation.

[0004] However, the video encoding and compression strategies in the aforementioned schemes are independent of the behavioral feedback generated by students after watching the videos. This makes it impossible to determine whether the image quality of a specific area in the video actually affects students' understanding of the teaching content contained in that area, nor can it establish a correlation between the problems reflected in students' learning feedback and the image quality processing strategies for specific areas of the screen. Consequently, the image quality processing of subsequent unplayed and review content lacks a targeted basis. Therefore, existing technologies suffer from the technical problem of insufficient intelligent processing of training videos due to the disconnect between video image quality processing strategies and student learning feedback, and the inability to make targeted adjustments to the image quality processing of specific areas based on student feedback data. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for intelligent processing of training videos to solve the problem of insufficient intelligent processing of training videos in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for intelligent processing of training videos, comprising: Get training videos and a timeline of key points; Frame segmentation and content recognition are performed on the training video, and the image regions that carry the teaching content within the corresponding time period of the knowledge point timeline are taken as key regions; The outer region of the key region is defined as the edge region, and the image region adjacent to the edge region but not marked as a key region is defined as the non-key region. A first and a second encoded reconstruction version are generated based on key regions, edge regions, and non-key regions. The first encoded reconstruction version reduces the fidelity of edge regions and non-key regions, while the second encoded reconstruction version increases the fidelity of key regions and edge regions. The fidelity is determined by the encoding parameters and reconstruction parameters. After playing the corresponding time periods of the first and second encoded reconstructed versions, the number of pauses, replays, and incorrect answers are collected as response data within the preset response time window; By comparing the response data of the first and second encoded reconstruction versions, regional image quality correlation anomalies are identified, and regions with different fidelity levels between the first and second encoded reconstruction versions are identified as image quality correlation regions. Update the encoding protection area range of the corresponding knowledge point based on the image quality associated area, and adjust the encoding parameters and reconstruction parameters of the area within the encoding protection area range in the unplayed video frames and review video segments.

[0007] Optionally, the teaching content includes blackboard writing, formulas, courseware text, and practical actions; Frame segmentation and content recognition are performed on the training videos, and the image regions carrying the teaching content within the corresponding time periods of the knowledge point timeline are identified as key regions, including: The training video is segmented into multiple video frames. Text recognition and target detection are performed on multiple video frames within the time period corresponding to the knowledge point timeline to obtain the image region carrying the teaching content. Image regions with a confidence level greater than or equal to a preset confidence threshold are identified as critical regions.

[0008] Optionally, the outer region of the key region is defined as the edge region, and the image region adjacent to the edge region but not marked as a key region is defined as the non-key region, including: The minimum bounding rectangle of the key area is determined as the outer expansion reference. The outer expansion distance is determined according to the preset video frame size ratio. The outer expansion area is obtained by expanding the outer expansion reference according to the outer expansion distance, and the outer expansion area is used as the edge area. Image regions whose boundary distance from the edge region is less than or equal to a preset adjacency distance and which are not marked as key regions are identified as non-key regions.

[0009] Optionally, the encoding parameters include bitrate allocation parameters and quantization parameters, and the reconstruction parameters include edge enhancement parameters and super-resolution reconstruction parameters; A first and a second encoded reconstruction version are generated based on key regions, edge regions, and non-key regions, including: Configure corresponding encoding and reconstruction parameters for key regions, edge regions, and non-key regions respectively, and obtain the fidelity level by mapping the encoding and reconstruction parameters; Keeping the encoding and reconstruction parameters of the critical regions unchanged, and reducing the fidelity of the edge and non-critical regions, the first encoded and reconstructed version is generated; Improve the fidelity of critical and edge regions, while keeping the encoding and reconstruction parameters of non-critical regions unchanged, to generate a second encoded and reconstructed version.

[0010] Optionally, after playing the corresponding time periods of the first and second encoded reconstructed versions, the number of pauses, replays, and incorrect answers are collected as response data within a preset response time window, including: Determine the corresponding knowledge point time periods for the first and second code reconstruction versions based on the knowledge point timeline. After the end of the knowledge point period, a preset reaction time window is activated. The preset reaction time window is determined based on the duration of the knowledge point period. Within a preset response time window, the number of pauses, replays, and incorrect answers are collected as response data.

[0011] Optionally, by comparing the response data of the first encoded reconstructed version and the second encoded reconstructed version, regional image quality correlation anomalies can be identified, including: Calculate the data difference between the first and second encoded reconstructed versions of the same type of response item data; Response item data whose data difference is greater than or equal to the corresponding preset difference threshold is identified as target response item data. The preset difference threshold is determined based on the historical average of the corresponding response item data. When the target response data of the first encoded reconstruction version is greater than the target response data of the second encoded reconstruction version, the anomaly type is determined to be a regional image quality association anomaly.

[0012] Optionally, the method further includes: To obtain the logical relationships between knowledge points; When there is no target response item data, the prerequisite knowledge points that have a dependency relationship with the knowledge points corresponding to the first and second encoding reconstruction versions are determined according to the logical relationship. When the correct answer rate for the current knowledge point is less than the preset correct answer rate threshold, the anomaly type is determined to be a content comprehension-related anomaly, and the coding protection area range of the knowledge point corresponding to the first and second coding reconstruction versions remains unchanged.

[0013] Optionally, the encoding protection region range of the corresponding knowledge point is updated according to the image quality-related region, and the encoding parameters and reconstruction parameters of the region within the encoding protection region range in the unplayed video frame and the review video segment are adjusted, including: When the image quality associated region only includes the critical region, the range of the encoding protection region is updated to cover the critical region. When the image quality associated region includes the edge region but does not include the non-critical region, the range of the encoding protection region is updated to cover the critical region and the edge region. When the image quality associated region includes the non-critical region, the range of the encoding protection region is updated to cover the critical region, the edge region and the non-critical region. The unplayed video frames after the playback progress that caused the regional image quality correlation anomaly and the review video segments generated according to the corresponding time period of the knowledge point timeline are taken as the adjustment objects. The encoding parameters and reconstruction parameters of the regions within the encoding protection area of ​​the adjustment objects are adjusted respectively, and the fidelity level is improved.

[0014] Secondly, this application provides a training video intelligent processing system, comprising: The acquisition module is used to acquire training videos and a timeline of knowledge points; The determination module is used to perform frame segmentation and content recognition on training videos, and to identify the image regions that carry teaching content within the time period corresponding to the knowledge point timeline as key regions. The determination module is also used to define the outer region of the key region as the edge region and the image region adjacent to the edge region but not marked as the key region as the non-key region. The generation module is used to generate a first encoded reconstruction version and a second encoded reconstruction version based on key regions, edge regions and non-key regions. The first encoded reconstruction version reduces the fidelity of edge regions and non-key regions, and the second encoded reconstruction version increases the fidelity of key regions and edge regions. The fidelity is determined by the encoding parameters and reconstruction parameters. The generation module is also used to collect the number of pauses, replays, and incorrect answers as response data within a preset reaction time window after playing the corresponding time periods of the first and second encoded reconstructed versions. The determination module is also used to compare the response data of the first encoded reconstruction version and the second encoded reconstruction version to determine the abnormal regional image quality association, and to determine the regions with different fidelity levels between the first encoded reconstruction version and the second encoded reconstruction version as image quality association regions. The adjustment module is used to update the encoding protection area range of the corresponding knowledge points based on the image quality associated area, and to adjust the encoding parameters and reconstruction parameters of the region within the encoding protection area range in the unplayed video frames and review video segments.

[0015] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor, configured to implement the steps of the training video intelligent processing method as described in the first aspect above when executing the computer program.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the training video intelligent processing method described in the first aspect above.

[0017] The intelligent training video processing method provided in this application addresses the problem that encoding and compression strategies and student learning feedback are independent in existing training video processing. By dividing the training video screen into key areas, edge areas, and non-key areas, two encoded and reconstructed versions are generated with different fidelity levels applied to different areas. After students actually play the video, response data such as the number of pauses, replays, and incorrect answers are collected and compared to identify the correlation between the differences in image quality between the two versions and the differences in student learning feedback.

[0018] This approach pinpoints the issues raised in student feedback to specific image quality-related areas within the video frame, thereby updating the encoding protection area for the corresponding knowledge point and making targeted adjustments to the encoding and reconstruction parameters for that area in unplayed video frames and review video segments. This application addresses the technical problem in existing technologies where video quality processing strategies are disconnected from student learning feedback and where targeted adjustments to image quality processing for specific areas are impossible.

[0019] Furthermore, to address the issue of diverse teaching content types and difficulty in ensuring accurate content area identification in training videos, this application performs text recognition and target detection on video frames within the corresponding time period of the knowledge point timeline. It identifies image areas in the video frames that carry teaching content, such as blackboard writing, formulas, courseware text, and practical actions, and determines key areas based on confidence thresholds, so that the delineation of key areas corresponds to the areas in the video frame that actually carry teaching content. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a training video intelligent processing method provided in an embodiment of this application; Figure 2 This application provides a schematic diagram of the structure for dividing a training video frame into different regions. Figure 3 A flowchart illustrating the process of updating the range of the coded protection region and adjusting parameters, provided for an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a training video intelligent processing system provided in an embodiment of this application. Detailed Implementation

[0022] In existing training video processing solutions, the formulation of video encoding and compression strategies is independent of the behavioral feedback generated by students after watching the videos: the configuration of encoding parameters relies on engineering experience or fixed rules, and different areas of the video screen are processed uniformly, failing to distinguish between areas carrying core teaching content and secondary areas such as the background; learning analysis relies solely on behavioral data such as pause frequency and replay frequency to help teachers understand learning, and there is a lack of data-level correlation mechanism between the two.

[0023] This disconnect makes it impossible to determine whether the image quality of specific areas in the video has a real impact on students' understanding of the corresponding teaching content, and it also makes it impossible to convert student feedback data into a basis for targeted adjustments to the image quality of unplayed and review content. As a result, the image quality assurance of training videos has long lacked the ability to dynamically adjust based on student learning outcomes.

[0024] To address the aforementioned issues, this application proposes an intelligent processing method for training videos. Its core lies in combining the division of video screen regions with the collection of student playback feedback data to establish a correspondence between the image quality status of screen regions and student learning feedback.

[0025] Specifically, this application first divides the video frame into a key area carrying the teaching content, an edge area formed by the expansion of the key area, and an adjacent non-key area. Then, it generates two encoded and reconstructed versions with different fidelity levels for different areas and plays them. Within a preset reaction time window after playback, it collects the number of pauses, replays, and incorrect answers from students. By comparing the response data corresponding to the two versions, it identifies the image quality-related areas that have an actual impact on students' learning outcomes. Then, it updates the encoded protection area range of the corresponding knowledge point and adjusts the encoding and reconstruction parameters of the corresponding areas in the unplayed video frames and review video segments accordingly.

[0026] This method incorporates students' actual learning feedback data into the video quality processing decision-making process, ensuring that the determination of the coded protection area and the adjustment of parameters are based on changes in learning outcomes that students can perceive. This solves the technical problem in existing technologies where video quality processing strategies are disconnected from student learning feedback and cannot be specifically adjusted for the quality processing of specific areas.

[0027] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] To address the problems of the prior art, embodiments of this application provide a method, apparatus, device, computer storage medium, and computer program product for intelligent processing of training videos. The intelligent processing method for training videos provided in this application embodiment will be described first below.

[0029] Figure 1 A flowchart illustrating an embodiment of the intelligent processing method for training videos provided in this application is shown. Figure 1 As shown, the method includes: S101. Obtain training videos and a timeline of key knowledge points.

[0030] Training videos refer to video files used to deliver teaching content in educational and training scenarios. They include various forms such as recorded course videos, practical demonstration videos, and blackboard explanation videos. The teaching content carried in the video can include blackboard writing, formulas, courseware text, and practical actions.

[0031] The knowledge point timeline refers to structured data that records the time periods corresponding to each knowledge point in the training video. Each record can include a knowledge point identifier, the start time and end time of the knowledge point in the training video, and is used to establish a mapping relationship between each time period in the training video and the corresponding knowledge point.

[0032] First, training videos are retrieved from a pre-established course repository. This repository stores video files corresponding to each training course, saved as encoded video streams organized chronologically, with frame indexes created for each image time point. The frame indexes are used to locate the image data corresponding to each time point in the encoded video stream and do not include decoded images that can be directly used for content recognition. When performing frame segmentation on the retrieved training videos, the encoded video stream is decoded and extracted frame by frame according to the frame indexes and the frame rate of the training videos, resulting in multiple video frames. Simultaneously, after the course recording is completed, the course creators manually annotate and generate a knowledge point timeline. Each record in the knowledge point timeline can be represented as... ,in For the first The identifier of each knowledge point and These are the start and end times of the corresponding time period in the training video for that knowledge point.

[0033] S102. Perform frame segmentation and content recognition on the training video, and take the image area that carries the teaching content within the time period corresponding to the knowledge point timeline as the key area.

[0034] Optionally, the teaching content in step S102 includes blackboard writing, formulas, courseware text, and practical actions.

[0035] Step S102, which involves frame segmentation and content recognition of the training video, and identifying the image regions carrying the teaching content within the corresponding time periods of the knowledge point timeline as key regions, can specifically include: S1021. Perform frame segmentation on the training video to obtain multiple video frames. Perform text recognition and target detection on multiple video frames within the time period corresponding to the knowledge point timeline to obtain the image region carrying the teaching content.

[0036] A video frame refers to a single frame of image data obtained by decoding the encoded video stream of a retrieved training video and extracting it frame by frame. Each video frame corresponds to a point in time in the training video and includes complete image information for that point in time. Text recognition refers to the process of detecting and recognizing text content in video frames using optical character recognition technology. It can recognize teaching content such as blackboard writing, formulas, and courseware text.

[0037] Object detection refers to the process of locating and classifying specific objects in video frames using a trained deep learning object detection model. It can detect objects such as practical actions and teaching tools. An image region carrying teaching content refers to a rectangular or pixel-level region within a video frame that includes the teaching content, obtained after text recognition or object detection. Each image region can be described by its location coordinates, region category, and recognition confidence score.

[0038] The training samples for the object detection model are derived from training video frame samples, which include single-frame images extracted from recorded course videos, practical demonstration videos, and whiteboard explanation videos. Each training sample is labeled with the object's bounding box, object category, and whether it contains teaching content. Object categories include practical actions, teaching tools, teacher hand gestures, and demonstration objects. Labeling quality is determined through independent labeling and review by two people. The cross-union ratio (CIU) of two labeled bounding boxes for the same object is considered satisfactory. If the target category is consistent, the label is considered valid; otherwise, the reviewer will re-label it.

[0039] Before training, the training samples are normalized in size and brightness, randomly cropped, compressed, perturbed, and slightly blurred. The object detection model includes a feature extraction network, a feature fusion network, and a detection head. The detection head outputs the target region location, target category, and recognition confidence. The loss function used during training is: ,in, For the existence loss of the target, For the target class cross-entropy loss, The regression loss is the bounding box of the target object. For the corresponding weight coefficients and satisfying .

[0040] The AdamW optimizer was used during training, with an initial learning rate of [missing information]. The batch size is 16, and the maximum number of training epochs is 100. Training stops when the average accuracy of object detection on the validation set fails to improve for 10 consecutive epochs. After training, the class probability output by the object detection model is used as the recognition confidence level.

[0041] First, the video file is extracted frame by frame according to the frame rate of the training video, resulting in multiple video frames. Let the total number of frames in the training video be... Then the set of video frames can be obtained. ,in Indicates the first Frame video frame.

[0042] Next, based on the start and end times of each knowledge point in the knowledge point timeline, video frames within the corresponding time periods are selected. Specifically, for each knowledge point... corresponding time period Extract all video frames within that time period as the set of frames to be identified. ,in For video frames The corresponding time point.

[0043] Then, text recognition and object detection are performed on each video frame in the set of frames to be recognized. The text recognition process uses optical character recognition (OCR) technology to detect blackboard writing, formulas, and courseware text in the video frames. For each video frame... Output the set of detected text regions, where each text region can be represented as... ,in The coordinates of the top left corner of the region. and For the area width and height, It can be a region category and can include blackboard writing, formulas, or courseware text. The confidence level for identifying this region.

[0044] The object detection process uses a deep learning-based object detection model to detect objects such as actions in video frames, and outputs a set of detected object regions. The representation of each object region is the same as that of a text region. Finally, all regions obtained from text recognition and object detection are merged to obtain a set of image regions carrying the teaching content. .

[0045] S1022. Determine the image regions with a confidence level greater than or equal to a preset confidence threshold as key regions.

[0046] Confidence score refers to the model's assessment of the correctness of the recognition result during text recognition or object detection. The value ranges from 0 to 1, with higher confidence scores indicating greater confidence in the recognition result. The preset confidence threshold is the lower limit of confidence used to filter image regions and can be set differently according to different teaching content categories. The preset confidence thresholds are shown in Table 1 below. Table 1: Comparison Table of Preset Reliability Thresholds

[0047] Table 1 shows the preset reliability threshold values ​​for different categories of teaching content. Determined through a manually labeled validation sample set. For each region category, different candidates are statistically analyzed. Lower image area The screening results were used to calculate the accuracy rate based on the manual annotation results. and recall rate Further calculations .

[0048] Will make The largest candidate value is determined as the pre-set confidence threshold corresponding to the region category. When multiple candidate values ​​correspond to When the difference is less than 0.01, the candidate value with the larger value is selected to reduce the probability that a falsely detected region is identified as a critical region. The values ​​in Table 1 are the default thresholds determined in the above manner.

[0049] First, the pre-set confidence thresholds are determined from Table 1 based on the category of the image regions. Next, each region in the set of image regions carrying the teaching content is filtered, and the confidence thresholds are set accordingly. Preset reliability thresholds for the corresponding categories If a comparison is made, If the condition is met, the image region is identified as a key region. Finally, all image regions that meet the criteria are grouped into a key region set. .

[0050] This embodiment performs frame segmentation and content recognition on the training video, and locates the video frames within the corresponding time period of each knowledge point according to the knowledge point timeline. It can accurately identify the image areas in the video screen that carry teaching content such as blackboard writing, formulas, courseware text, and practical actions, avoiding indiscriminate processing of the entire video screen.

[0051] Meanwhile, by setting different pre-set confidence thresholds for different categories of teaching content, false detection areas with low recognition confidence can be filtered out, ensuring that the image areas identified as key areas correspond to the areas in the video that actually carry the teaching content. This allows the image quality protection strategy to be applied precisely to the key areas that truly carry the teaching content.

[0052] S103. The outer region of the key region is the edge region, and the image region adjacent to the edge region but not marked as a key region is the non-key region.

[0053] Optionally, step S103, which uses the outer region of the key region as the edge region and the image region adjacent to the edge region but not marked as a key region as a non-key region, may specifically include: Figure 2 This illustration shows a schematic diagram of the structure of the training video frame area division provided in one embodiment of this application. For example... Figure 2 As shown, the method includes: S1031. Determine the minimum bounding rectangle of the key area as the outer expansion reference, determine the outer expansion distance according to the preset video frame size ratio, and expand the outer expansion reference according to the outer expansion distance to obtain the outer expansion area as the edge area.

[0054] The minimum bounding rectangle is the smallest rectangle that completely encloses the critical region, and its sides are parallel to the horizontal and vertical directions of the video frame. The expansion reference is the reference boundary used for expansion operations and can be the four sides of the minimum bounding rectangle.

[0055] The outward extension distance refers to the pixel distance extending outward from the outward extension reference, and this distance is determined according to the aspect ratio of the video frame to accommodate video frames of different resolutions. The edge region refers to the annular region between the outward extension region obtained after extending the outward extension distance from the outward extension reference and the key region, and this region usually includes the background, teacher gestures, and other auxiliary teaching content surrounding the key region. The preset video frame aspect ratios are shown in Table 2 below: Table 2: Preset Video Frame Size Ratio Comparison Table

[0056] As shown in Table 2, Table 2 presents the preset video frame size ratios corresponding to different video frame resolutions. The coverage distance of auxiliary teaching content surrounding key areas in manually annotated samples is determined. For widths of... Height is From the video frames, statistically analyze the distance samples between the outer boundary of the key area and the outer boundary of the auxiliary teaching content, and take the 80th percentile of this distance sample as the reference extended distance. And then according to Reverse determination of the corresponding resolution The values ​​in Table 2 are the default values ​​obtained by statistically analyzing training video samples of different resolutions in the manner described above.

[0057] Combination Figure 2 As shown, the logic behind dividing different regions of the video frame is further explained. First, the key region set... Each key region in the image is assigned its minimum bounding rectangle. In practice, since the image regions output by character recognition and object detection are themselves regular rectangles, the determined image region is its minimum bounding rectangle. Figure 2 Key areas marked with solid-line rectangles For example, its corresponding external reference can be represented as ,in These are the pixel coordinates of the top-left corner of the solid-line rectangle. and These are the pixel width and pixel height of the solid-line rectangle, respectively.

[0058] Next, determine the preset video frame size ratio according to Table 2. Let the pixel width of the entire video frame be... And the pixel height is Then the outward expansion distance The calculation formula is ,in The smaller of the video frame width and height is used for the outward expansion distance. exist Figure 2 The direction is clearly indicated by a double-headed arrow segment.

[0059] Then, according to the outward expansion distance Extend the circumscribed reference outwards, that is, shift the upper left corner coordinates upwards and to the left by a certain distance. The coordinates of the lower right corner are shifted downwards and to the right by a distance. Thus constructing Figure 2 The expanded rectangular area is enclosed by a large-spaced dashed rectangle. .

[0060] Finally, the expanded dashed rectangular area Key areas with solid lines inside The annular geometric region formed between them is defined as the corresponding edge region. , its in Figure 2 The middle part is represented as a buffer zone sandwiched between the solid-line frame and the large-spaced dashed-line frame. The set of all edge regions can be represented as... .

[0061] S1032. Determine image regions whose boundary distance from the edge region is less than or equal to a preset adjacency distance and which are not marked as key regions as non-key regions.

[0062] Boundary distance refers to the shortest pixel distance between the boundary of an image region and the boundary of an edge region, and this distance is used to determine whether an image region is adjacent to an edge region. Preset adjacency distance is a boundary distance threshold used to determine whether an image region is a non-critical region; only image regions with a boundary distance less than or equal to the preset adjacency distance are identified as non-critical regions.

[0063] Non-critical regions refer to image areas adjacent to edge regions but not marked as critical regions. These regions typically include background elements, decorations, and other content unrelated to the teaching content in the video frame. Preset adjacency distances are set differently for different region categories, as shown in Table 3 below: Table 3: Preset Adjacency Distance Reference Table

[0064] As shown in Table 3, Table 3 presents the preset adjacency distance values ​​for different region categories. Preset Adjacency Distance The boundary distance between candidate non-critical regions and edge regions is determined based on the distribution of the boundary distance in manually labeled samples.

[0065] For region categories Calculate the boundary distance between candidate non-critical regions and edge regions under this category. The 80th percentile of the boundary distance samples is taken as the preset adjacency distance for that category. Therefore, when the condition is satisfied... When this happens, the corresponding candidate non-critical regions are determined as non-critical regions. The values ​​in Table 3 are the default preset adjacency distances obtained by statistical analysis in the above manner.

[0066] After delineating the edge regions, the initial set of image regions carrying the teaching content is first... In the process, elements that have been marked as key regions are removed, and the remaining unmarked image regions are extracted to construct a candidate set of non-key regions. .

[0067] Next, combined Figure 2 As shown, for the set of candidate non-critical regions For each image region, the boundary spatial distance between it and existing edge regions is calculated. Specifically, for Figure 2 Any candidate non-critical region represented by a tightly dotted rectangle. and the already generated set of edge regions A certain edge region in Calculate the shortest pixel distance between the two boundaries. The shortest pixel distance exist Figure 2 The data is quantitatively labeled by a bidirectional arrow line segment that spans the outer edge of the large-spaced dashed frame and the inner edge of the dotted frame. The value is obtained by solving the shortest Euclidean distance between all pixel pairs on the boundary of the two rectangular regions.

[0068] Then, based on the candidate non-critical region For specific teaching content categories, retrieve the corresponding preset adjacency distances from Table 3. and the actual calculated shortest pixel distance A threshold comparison is performed with the preset adjacent distance. If at least one edge region exists in the entire image... Enough to satisfy the boundary distance If the constraints are met, the candidate non-critical region is determined to be spatially highly adjacent to the edge region, and is formally marked as a non-critical region. Finally, all candidate non-critical regions that passed the adjacency check are merged and combined to generate the final set of non-critical regions. .

[0069] This embodiment generates an edge region by expanding the key region outward. This allows a buffer area containing auxiliary teaching content to be defined around the key region. This edge region is adjacent to the key region and transitions to unrelated content such as the background in the video frame, thus avoiding obvious image quality abrupt changes when the key region and the background region are processed with completely different fidelity levels.

[0070] Meanwhile, by calculating the boundary distance between candidate non-critical regions and edge regions and comparing it with the preset adjacency distance, non-critical regions adjacent to edge regions can be accurately identified.

[0071] S104. Generate a first encoding reconstruction version and a second encoding reconstruction version based on the key region, edge region and non-key region. The first encoding reconstruction version reduces the fidelity of the edge region and non-key region, and the second encoding reconstruction version increases the fidelity of the key region and edge region. The fidelity is determined by the encoding parameters and reconstruction parameters.

[0072] Optionally, the encoding parameters in step S104 include bitrate allocation parameters and quantization parameters, and the reconstruction parameters include edge enhancement parameters and super-resolution reconstruction parameters.

[0073] Step S104, which generates the first and second encoded reconstruction versions based on key regions, edge regions, and non-key regions, may specifically include: S1041. Configure the corresponding encoding parameters and reconstruction parameters for key regions, edge regions and non-key regions respectively, and obtain the fidelity level by mapping the encoding parameters and reconstruction parameters.

[0074] Encoding parameters refer to the set of parameters that control the degree of compression and image quality preservation during video encoding, including bitrate allocation parameters and quantization parameters. Bitrate allocation parameters refer to the bitrate value allocated to a specific region; a higher value indicates more data allocated to that region, resulting in better image quality preservation.

[0075] Quantization parameters are numerical values ​​that control the quantization step size; the smaller the value, the higher the quantization accuracy and the less image quality loss. Reconstruction parameters are a set of parameters used to restore and enhance image quality during video decoding and reconstruction, including edge enhancement parameters and super-resolution reconstruction parameters.

[0076] Edge enhancement parameters control the degree of edge sharpening; a higher value indicates a more significant enhancement of image edges. Super-resolution reconstruction parameters control the reconstruction intensity from low resolution to high resolution; a higher value indicates a higher reconstructed resolution. Fidelity level is a numerical level obtained by mapping encoding and reconstruction parameters and is used to quantify the degree of image quality preservation in a video region; a higher fidelity level indicates that the image quality of that region is closer to the original video.

[0077] First, the key area set Edge region set Non-critical area set Each region is configured with separate encoding and reconstruction parameters. For any given region... Its encoding parameters can be expressed as ,in Assign parameters to the bitrate. These are the quantization parameters. The reconstruction parameters can be expressed as... ,in For edge enhancement parameters, These are the parameters for super-resolution reconstruction.

[0078] Next, different parameter values ​​are configured according to the region type. For critical regions, higher bitrate allocation parameters and lower quantization parameters are configured to ensure image quality, while moderate edge enhancement parameters and super-resolution reconstruction parameters are configured. For edge regions, medium bitrate allocation parameters and quantization parameters are configured, along with moderate reconstruction parameters. For non-critical regions, lower bitrate allocation parameters and higher quantization parameters are configured to improve compression ratio, while lower reconstruction parameters are configured.

[0079] Then, the encoding parameters and reconstruction parameters are converted to fidelity levels using a mapping function. The mapping function can be expressed as: ,in For the region The fidelity level, , , and These are the maximum values ​​of each parameter, and are set in this embodiment. , , , , , , and The weights corresponding to each parameter and satisfying For example, in one implementation, set , , , Finally, the calculated fidelity levels are normalized to integer levels from 0 to 10 to obtain the initial fidelity levels for each region.

[0080] S1042. Keep the encoding and reconstruction parameters of the critical regions unchanged, and reduce the fidelity of the edge regions and non-critical regions to generate the first encoded reconstruction version.

[0081] The first encoding reconstruction version refers to an encoding reconstruction result generated for the same knowledge point time period. In this version, the key areas retain the original encoding parameters and reconstruction parameters, while the fidelity of the edge areas and non-key areas is reduced.

[0082] First, maintain the key area set. The encoding and reconstruction parameters remain unchanged for all regions, that is, for any key region Its encoding parameters and reconstruction parameters All settings remain at their initial configuration values. Next, the edge region set... Non-critical area set The fidelity level of certain areas is reduced.

[0083] Specifically, the fidelity level of edge and non-critical areas is reduced by a preset level, such as two levels. This reduction in fidelity is achieved by adjusting encoding and reconstruction parameters. For areas where fidelity reduction is necessary... Reduce its bitrate allocation parameters Increase quantization parameters Both edge enhancement parameters and super-resolution reconstruction parameters were reduced. .

[0084] Then, based on the adjusted encoding parameters, the knowledge points... Video frames corresponding to the specified time period are regionally encoded. For video frames... The key regions in the frame are encoded using unchanged encoding parameters, while the edge regions and non-key regions are encoded using reduced encoding parameters, resulting in the encoded video stream.

[0085] Next, the encoded video stream is decoded, and reconstruction processing is performed according to the reconstruction parameters corresponding to each region. For critical regions, edge enhancement and super-resolution reconstruction are performed using the unchanged reconstruction parameters; for edge regions and non-critical regions, reconstruction is performed using the reduced reconstruction parameters. Finally, the reconstructed regions are combined into a complete video frame to obtain the knowledge points. The corresponding first encoded reconstructed version .

[0086] S1043. Improve the fidelity of critical and edge regions, while keeping the encoding and reconstruction parameters of non-critical regions unchanged, and generate a second encoded and reconstructed version.

[0087] The second encoding reconstruction version refers to another encoding reconstruction result generated for the same knowledge point time period. In this version, the fidelity of key and edge regions is improved, while the original encoding and reconstruction parameters are maintained for non-key regions.

[0088] First, maintain the set of non-critical areas. The encoding and reconstruction parameters remain unchanged for all regions, meaning that for any non-critical region... Its encoding parameters and reconstruction parameters All settings remain at their initial configuration values. Next, the key region set... and edge region set The fidelity level of the region is improved.

[0089] Specifically, the fidelity level of key and edge areas is increased by a preset level, such as two levels. This increase in fidelity is achieved by adjusting encoding and reconstruction parameters. For areas requiring a higher fidelity level... Increase its bitrate allocation parameters Quantization parameters reduced Both edge enhancement parameters and super-resolution reconstruction parameters are increased. .

[0090] Then, based on the adjusted encoding parameters, the knowledge points... Video frames corresponding to the specified time period are regionally encoded. For video frames... The key and edge regions in the frame are encoded using the improved encoding parameters, while the non-key regions are encoded using the unchanged encoding parameters, resulting in the encoded video stream.

[0091] Next, the encoded video stream is decoded, and reconstruction processing is performed according to the reconstruction parameters corresponding to each region. For critical and edge regions, edge enhancement and super-resolution reconstruction are performed according to the improved reconstruction parameters; for non-critical regions, reconstruction is performed using the unchanged reconstruction parameters. Finally, the reconstructed regions are combined into a complete video frame to obtain the knowledge points. The corresponding second-encoded reconstruction version .

[0092] This embodiment configures differentiated encoding and reconstruction parameters for key regions, edge regions, and non-key regions, and maps them to fidelity levels. This enables precise image quality control for different regions in the video frame, avoiding the application of a uniform encoding strategy to the entire frame.

[0093] Meanwhile, by generating two video versions with different fidelity levels—the first encoded reconstruction version and the second encoded reconstruction version—the fidelity level of the edge and non-critical areas is reduced while the fidelity level of the critical areas remains unchanged, while the fidelity level of the critical and edge areas is increased while the fidelity level of the non-critical areas remains unchanged. This allows for the accurate identification of specific areas where changes in image quality have a real impact on students' learning outcomes.

[0094] S105. After playing the corresponding time periods of the first and second encoded reconstructed versions, collect the number of pauses, replays, and incorrect answers within the preset response time window as response data.

[0095] Optionally, after playing the first and second reconstructed versions for the corresponding time periods in step S105, the process of collecting the number of pauses, replays, and incorrect answers as response data within a preset reaction time window may specifically include: S1051. Determine the knowledge point time periods corresponding to the first and second code reconstruction versions based on the knowledge point time axis.

[0096] The knowledge point time period refers to the time period from the start time to the end time of the knowledge point corresponding to the first and second encoding reconstruction versions in the training video. This time period is determined by the record of the corresponding knowledge point in the knowledge point timeline. The duration of the knowledge point time period can be the difference between the end time and the start time.

[0097] First, from the knowledge point timeline Searching for and reconstructing the first encoded version Second encoding reconstruction version The corresponding knowledge point records. Next, based on the found knowledge point records, the start time of each knowledge point is extracted. and end time , time period The designated knowledge point time period is defined as follows: The duration of each knowledge point time period is... .

[0098] S1052. After the end of the knowledge point period, start the preset reaction time window. The preset reaction time window is determined according to the duration of the knowledge point period.

[0099] The preset response time window refers to the time interval for collecting student response data, which starts after the end of the knowledge point period. The duration of the preset response time window is determined according to the duration of the knowledge point period in a preset ratio to adapt to the student response cycle corresponding to different lengths of knowledge point periods.

[0100] First, determine the duration of the preset reaction time window. The duration of the preset reaction time window... Based on the duration of the knowledge point period According to the preset ratio Calculated, i.e. The preset ratio The value can be set according to the actual teaching scenario, such as 0.3, which means the duration of the preset reaction time window is 30% of the duration of the knowledge point segment. Then, at the end of the knowledge point segment... Then, immediately activate the preset reaction time window, which covers a time range of [time range to be filled in]. .

[0101] S1053. Collect the number of pauses, replays, and incorrect answers as response data within the preset response time window.

[0102] Response data refers to a set of behavioral data collected from students within a preset response time window, reflecting their understanding of knowledge points. This includes the number of pauses, replays, and incorrect answers. The number of pauses refers to the cumulative number of times a student pauses the replay video within the preset response time window (i.e., after the knowledge point segment has finished playing and the test or summary section has begun, due to poor image quality of the preceding content), and actively clicks to replay. A higher value generally indicates that the student has difficulty understanding the knowledge point.

[0103] The number of replays refers to the cumulative number of times a student clicks a specific button within the preset reaction time window to replay the previously completed knowledge point segment. A higher value usually indicates that the student needs to repeatedly watch the knowledge point due to factors such as previous image quality. The number of incorrect answers refers to the number of questions a student answers incorrectly when answering test questions related to that knowledge point within the preset reaction time window. A higher value usually indicates that the student has a lower level of mastery of that knowledge point. The specific forms of test questions include, but are not limited to, objective multiple-choice questions, true / false questions, or fill-in-the-blank questions.

[0104] First, within the preset reaction time window Within the system, continuous monitoring of student actions and answer results is performed. For pause counts, the system records the moment each pause is triggered within a preset reaction time window, and the total number of pauses is accumulated to obtain the total pause count. .

[0105] For the number of replays, the time when the student triggers a replay operation within the preset reaction time window is recorded, and the number of all replay operations is accumulated to obtain the replay count. Next, regarding the number of incorrect answers, when students submit their answers within the preset reaction time window, the system will consider the number of incorrect answers related to the knowledge points. When answering relevant test questions, the system automatically compares the student's submitted answers with the standard answers according to a preset objective scoring standard, counts the number of questions with inconsistent answers, and obtains the number of incorrect answers. .

[0106] Then, the number of pauses Number of replays and the number of incorrect answers Combined into response item data Record the first encoded reconstructed version respectively Second encoding reconstruction version Corresponding response item data and .

[0107] This embodiment locates the knowledge point time periods corresponding to the first and second encoded reconstruction versions based on the knowledge point time axis, and determines a preset reaction time window proportionally after the knowledge point time period ends. This can limit the collection of students' operational behavior and answer results to a time range directly related to the knowledge points, avoiding the collection of interference data unrelated to the knowledge points.

[0108] At the same time, by recording the response data corresponding to the first and second coded reconstruction versions respectively, an independent and corresponding data basis can be provided for comparing the differences in student responses between the two versions.

[0109] S106. Compare the response data of the first encoded reconstruction version and the second encoded reconstruction version to determine the abnormal regional image quality association, and identify the regions with different fidelity levels between the first encoded reconstruction version and the second encoded reconstruction version as image quality association regions.

[0110] Regional image quality correlation anomaly refers to an abnormal state determined by comparing the response data of the first and second encoded reconstruction versions. It indicates that there is a significant correlation between the difference in fidelity between the two versions and the students' learning behavior feedback, that is, the change in fidelity has an observable actual impact on the students' understanding of the knowledge point.

[0111] Image quality associated regions refer to regions with different fidelity levels in the first and second encoded reconstruction versions, and these regions are identified as the source of image quality changes that cause abnormal image quality association in the region. Image quality associated regions can be the union of one or more regions among critical regions, edge regions, or non-critical regions.

[0112] First, rebuild the first encoded version. Second encoding reconstruction version Corresponding response item data and A comparison was made, calculating the data differences for three types of response items: the number of pauses, the number of replays, and the number of incorrect answers. It was then determined whether the response item data for the first encoded reconstructed version was significantly greater than the data for the same type of response item for the second encoded reconstructed version. If at least one type of response item met the above condition, then an anomaly in regional image quality correlation was identified. Detailed judgment methods for the comparison process will be further explained in the subsequent weighting section.

[0113] Next, after identifying the abnormal correlation in regional image quality, the first encoded reconstruction version was determined. Second encoding reconstruction version Regions with different fidelity levels. Specifically, comparing key region sets one by one. Edge region set Non-critical area set The fidelity levels of each region in the two versions are analyzed, and regions where the fidelity levels differ are extracted.

[0114] When generating the first encoded reconstruction version, the fidelity of edge and non-critical regions is reduced while that of critical regions remains unchanged; when generating the second encoded reconstruction version, the fidelity of critical and edge regions is increased while that of non-critical regions remains unchanged. Therefore, the regions with different fidelity levels between the two versions can be one or more of the edge, non-critical, and critical regions.

[0115] Then, all areas with different fidelity levels were identified as image quality-related areas. If there is only one area with different fidelity levels, then that area will be directly identified as the image quality-related area. If there are multiple regions with different fidelity levels, then the union of these regions with different fidelity levels is defined as the image quality-related region. .

[0116] This embodiment compares the response data corresponding to the first and second encoded reconstruction versions, and can determine whether the changes in image quality have an observable impact on the students' understanding of knowledge points based on the students' actual learning behavior data, thus avoiding setting image quality protection strategies based solely on subjective experience.

[0117] At the same time, by identifying the areas with different fidelity levels between the two versions as image quality-related areas, it is possible to accurately pinpoint the problems reflected in student learning feedback to specific areas in the video frame.

[0118] Optionally, the process of comparing the response data of the first encoded reconstructed version and the second encoded reconstructed version in step S106 to determine the regional image quality correlation anomaly may specifically include: S1061. Calculate the data difference between the first and second encoded reconstructed versions of the same type of response item data.

[0119] The data difference refers to the difference between the data of the same type of response item corresponding to the first and second coded reconstruction versions. This difference reflects the degree of difference between the two versions in student learning behavior feedback. The larger the data difference, the more significant the difference in student response to this type of response item between the two versions.

[0120] First, extract the first encoded reconstructed version. Second encoding reconstruction version Corresponding response item data and Next, the data differences are calculated for each of the three types of response items; specifically, the difference in the number of pauses is calculated. Difference in number of replays Difference in the number of incorrect answers Finally, the three types of data differences are combined into a data difference set. .

[0121] S1062. Target response data are determined when the data difference is greater than or equal to the corresponding preset difference threshold. The preset difference threshold is determined based on the historical average of the corresponding response data.

[0122] The preset difference threshold is a threshold used to determine whether the data difference is significant. This threshold is determined based on the historical average of the corresponding type of response item data. Only response item data with a data difference greater than or equal to the preset difference threshold is determined as target response item data.

[0123] Target response item data refers to response item data whose data difference meets the preset difference threshold condition, and this data indicates that there is a significant difference between the two versions in the corresponding type of response item. Table 4 shows the method for determining the preset difference threshold for different response item types, as shown in Table 4 below: Table 4: Comparison Table of Preset Difference Threshold Determination Methods

[0124] As shown in Table 4, Table 4 presents the methods for determining the preset difference thresholds for different response item types. (Preset difference threshold for number of pauses) Preset difference threshold for playback times The preset difference threshold between the number of incorrect answers and the number of questions. All are determined based on historical averages.

[0125] Specifically, the historical average number of pauses, replays, and incorrect answers for the same knowledge point type in past training courses were statistically analyzed. The preset difference thresholds for the number of pauses and replays were set to 0.5 times the corresponding historical averages, and the preset difference threshold for the number of incorrect answers was set to 0.3 times the corresponding historical averages.

[0126] First, determine the corresponding preset difference threshold from Table 4 based on the historical average of the response data for each type. The historical average refers to the average value of the same type of response data corresponding to the same knowledge point in past training courses. The preset difference threshold can be set as a certain percentage of the historical average, such as 0.5 times the historical average, to exclude the interference of fluctuations in normal learning behavior.

[0127] Next, the data difference set Each component in the table is compared with a preset difference threshold for its corresponding type. Let the preset difference threshold for the number of pauses be... The preset threshold for the difference in the number of replays is... The preset threshold for the difference in the number of incorrect answers is... Judge separately , and Whether it holds true. Finally, the response item data that meets the above conditions is determined as the target response item data and combined into the target response item data set. .

[0128] S1063. When the target response item data of the first encoding and reconstruction version is greater than the target response item data of the second encoding and reconstruction version, the anomaly type is determined to be a regional image quality association anomaly.

[0129] Anomaly type refers to the classification result of the source of student learning difficulties determined based on the comparison results of response item data, and can be either regional image quality correlation anomaly or content comprehension correlation anomaly. Regional image quality correlation anomaly refers to a situation where the value corresponding to the first encoded reconstructed version of the target response item data is greater than the value corresponding to the second encoded reconstructed version. This indicates that the processing method of reducing the fidelity level of edge areas and non-critical areas has a negative impact on students' understanding of knowledge points, that is, there is a direct correlation between the reduction in image quality and students' learning difficulties.

[0130] First, the target response item data set For each type of target response item data, the first encoded reconstructed version is extracted. Second encoding reconstruction version The corresponding values ​​are compared. Next, the target response item dataset is analyzed. Perform logical verification to determine whether the value of the target response item data corresponding to the first encoded reconstructed version is greater than the value corresponding to the second encoded reconstructed version. If the target response item data set... If the data of the target response item corresponding to the first encoding and reconstruction version is not empty and is greater than the data of the same type corresponding to the second encoding and reconstruction version, then the anomaly type is determined to be a regional image quality association anomaly.

[0131] Optionally, the method in step S106 further includes: Obtain the logical relationships between knowledge points. When target response data is unavailable, determine the prerequisite knowledge points that are dependent on the knowledge points corresponding to the first and second reconstructed encoding versions based on the logical relationships. If the correct answer rate for the prerequisite knowledge points is less than a preset correct answer rate threshold, determine the anomaly type as a content comprehension association anomaly, and maintain the encoding protection area of ​​the knowledge points corresponding to the first and second reconstructed encoding versions unchanged.

[0132] The logical relationship between knowledge points refers to structured data that records the prerequisite dependencies between knowledge points. Each record can include a knowledge point identifier and the identifier of the prerequisite knowledge point that has a prerequisite dependency relationship with that knowledge point. It is used to determine whether a student's learning difficulties stem from insufficient understanding of the prerequisite knowledge points when there is no target response data.

[0133] Prerequisite knowledge points refer to knowledge points that are logically related to the current knowledge point, and the degree of mastery of these prerequisite knowledge points directly affects the understanding of the current knowledge point. The accuracy rate refers to the proportion of test questions answered correctly when students answer questions related to prerequisite knowledge points out of the total number of questions; the lower the ratio, the lower the student's mastery of the prerequisite knowledge points.

[0134] Content comprehension association anomaly refers to an abnormal state determined by judging the correct answer rate of prior knowledge points in the absence of target response data, indicating that the student's learning difficulties mainly stem from insufficient understanding of prior knowledge points rather than the image quality problem of the video area corresponding to the current knowledge point.

[0135] Preset accuracy threshold The accuracy rate for different knowledge point types was determined based on historical training data. For prerequisite knowledge points... The system identifies the knowledge point type, calculates the correct answer rate for that knowledge point in historical samples, and uses the 25th percentile of the correct answer rate distribution as the corresponding preset correct answer rate threshold. .

[0136] In this embodiment, the highest preset accuracy threshold for practical knowledge points is set at 0.80 because of the high requirements for operational accuracy; the next highest is set at 0.75 for calculation knowledge points; the preset accuracy threshold for conceptual knowledge points is set at 0.70; and the preset accuracy threshold for proof knowledge points is set at a relatively low 0.65 because of the high difficulty in understanding them.

[0137] First, obtain the logical connections between the knowledge points. ,in To be related to knowledge points Identify the prerequisite knowledge points that indicate dependencies. Next, determine the target response item data set. Is it empty? If the result is empty, it indicates that the difference in response data between the two versions is not significant. The student's learning difficulties may not be due to differences in image quality, and further investigation is needed to check the mastery of prerequisite knowledge points.

[0138] Then, based on logical relationships Search and knowledge points Prerequisites for understanding dependencies Extract the correct answer rate corresponding to the prerequisite knowledge points. The correct answer rate is the ratio of the number of questions answered correctly by students when answering test questions related to prerequisite knowledge points to the total number of questions.

[0139] Next, the corresponding preset accuracy threshold is determined based on the type of prerequisite knowledge points. Increase the accuracy of answering questions on prerequisite knowledge points Compared with the preset accuracy threshold, if If so, the exception type is determined to be a content understanding association exception, and the first encoded reconstructed version is maintained. Second encoding reconstruction version Corresponding knowledge points The encoding protection area remains unchanged, meaning the encoding parameters and reconstruction parameters for the area corresponding to the current knowledge point are not adjusted, and the problem is attributed to insufficient understanding of prior knowledge points. If If the student's learning behavior differences are within the normal fluctuation range, a normal status indicator is output and the current abnormal judgment process ends.

[0140] This embodiment calculates the difference in response data and compares it with a preset difference threshold. This allows for the filtering out of normal fluctuations in student responses between two versions, preventing accidental behavioral fluctuations from being misjudged as image quality-related anomalies. Furthermore, by introducing the logical relationships between knowledge points and the accuracy of answers to prerequisite knowledge questions, it can further differentiate the sources of student learning difficulties when no significant image quality-related anomalies are found, determining whether the anomaly stems from image quality issues or insufficient understanding of prerequisite knowledge points.

[0141] S107. Update the encoding protection area range of the corresponding knowledge point according to the image quality associated area, and adjust the encoding parameters and reconstruction parameters of the area within the encoding protection area range in the unplayed video frame and the review video segment.

[0142] Optionally, step S107, which updates the encoding protection region range of the corresponding knowledge point based on the image quality-related region and adjusts the encoding parameters and reconstruction parameters of the region within the encoding protection region range in the unplayed video frame and the review video segment, may specifically include: Figure 3 This illustration shows a flowchart of updating the coded protection region range and adjusting parameters according to an embodiment of this application. Figure 3 As shown, the method includes: S1071. When the image quality associated region only includes the critical region, the range of the encoding protection region is updated to cover the critical region. When the image quality associated region includes the edge region but does not include the non-critical region, the range of the encoding protection region is updated to cover the critical region and the edge region. When the image quality associated region includes the non-critical region, the range of the encoding protection region is updated to cover the critical region, the edge region and the non-critical region.

[0143] The encoded protection area refers to the range of video frame areas that require focused image quality protection based on specific knowledge points, and this range is dynamically updated according to the composition of the image quality-related areas. The encoded protection area can be the union of critical areas, critical areas, and edge areas, or the union of critical areas, edge areas, and non-critical areas, depending on the composition of the image quality-related areas.

[0144] First, extract the knowledge points. Corresponding image quality related areas ,judge Its composition. Specifically, to determine Does it include a set of key regions? Set of regions and edge regions Set of regional and non-critical regions The area in.

[0145] Next, based on the judgment result, the scope of the coded protection area is updated according to the following rules. If the image quality is related to the area If only critical areas are included, the scope of the coded protection area will be determined. Update to cover key area set If the image quality is related to the area If the edge area is included but non-critical areas are excluded, then the protected area range will be encoded. Update to cover key area set and edge region set The union of .

[0146] If image quality is related to the area If non-critical areas are included, the scope of the coded protection area will be determined. Update to cover key area set Edge region set Non-critical area set The union of the values ​​yields the updated coded protection region range. .

[0147] S1072. Taking the unplayed video frames after the playback progress that caused the regional image quality correlation abnormality and the review video segments generated according to the corresponding time period of the knowledge point time axis as the adjustment objects, the encoding parameters and reconstruction parameters of the regions within the encoding protection area of ​​the adjustment objects are adjusted respectively, and the fidelity level is improved.

[0148] Unplayed video frames refer to video frames that have not been played by students after the playback progress of the area with abnormal image quality association. These video frames correspond to the content after the current playback position in the training video. The corresponding areas in these frames need to be re-encoded and reconstructed according to the updated encoding protection area range.

[0149] Review video segments refer to video clips extracted or regenerated from training videos according to the corresponding time periods of the knowledge points on the timeline, which are used by students to review specific knowledge points. The time range of the review video segments is consistent with the time period of the corresponding knowledge points on the timeline.

[0150] First, identify the target of the adjustment. For unplayed video frames, obtain the real-time playback progress of the student's video at the moment when the identified area of ​​image quality correlation anomaly occurs. From video frame set Extracting this moment All subsequent video frames result in a set of unplayed video frames. For review video segments, based on the knowledge points in the timeline... corresponding time period By extracting video frames from the training videos for the corresponding time periods, a set of review video frames can be obtained. Set of unplayed video frames and a collection of review video clips and frames Merge into a set of adjustment objects .

[0151] Next, adjust the set of objects. For each video frame, determine the range of the updated coded protection region within that frame. The area within. Specifically, for video frames in the set of objects to be adjusted. ,Will The location coordinates of each region in the middle and Perform comparison and extract the location located The area within is designated as the area to be adjusted.

[0152] Then, the coding and reconstruction parameters of the region to be adjusted are progressively adjusted, with the bitrate allocation parameter of the region to be adjusted increasing each time. The preset step size is increased, and the quantization parameters are decreased each time. The preset step size is lowered, and the edge enhancement parameters and super-resolution reconstruction parameters are increased by a certain amount each time. The preset step size is increased so that the area to be adjusted is within the range of... arrive In the integer grading system, the fidelity level is increased by at least one level until the preset maximum fidelity level is reached. The adjusted encoding parameters are denoted as... The reconstruction parameters are denoted as And recalculate the fidelity level based on the mapping function. ,confirm Higher than the fidelity level before adjustment .

[0153] Finally, the regions to be adjusted in each frame of the set of objects to be adjusted are re-encoded according to the adjusted encoding parameters, and then reconstructed according to the adjusted reconstruction parameters to obtain the adjusted object video frames with improved regional fidelity within the encoded protection area.

[0154] This embodiment dynamically updates the range of the encoded protection area based on the composition of the image quality-related areas, which can accurately match the coverage of image quality protection to the specific areas that actually affect students' learning outcomes, avoiding redundant encoding overhead caused by uniformly applying high-fidelity processing to all areas.

[0155] At the same time, by applying the adjustment range to both unplayed video frames and review video segments, it can ensure that students can obtain high-quality images in areas directly related to image quality anomalies during subsequent learning and review processes. This provides targeted image quality assurance for key teaching content where students need clear image quality the most, and achieves dynamic image quality optimization driven by student learning feedback.

[0156] Figure 4 This is a schematic diagram illustrating a specific implementation of a training video intelligent processing system provided in this application. (Refer to...) Figure 4 The system may include: Module 410 is used to acquire training videos and knowledge point timelines; The determination module 420 is used to perform frame segmentation and content recognition on the training video, and to identify the image regions that carry the teaching content within the time period corresponding to the knowledge point timeline as key regions. The determination module 420 is further configured to use the outer region of the key region as the edge region and the image region adjacent to the edge region but not marked as the key region as the non-key region; The generation module 430 is used to generate a first encoded reconstruction version and a second encoded reconstruction version based on key regions, edge regions and non-key regions. The first encoded reconstruction version reduces the fidelity of edge regions and non-key regions, and the second encoded reconstruction version increases the fidelity of key regions and edge regions. The fidelity is determined by the encoding parameters and reconstruction parameters. The generation module 430 is also used to collect the number of pauses, replays, and the number of incorrect answers as response data within a preset reaction time window after playing the corresponding time periods of the first and second encoded reconstructed versions. The determination module 420 is also used to compare the response data of the first encoded reconstruction version and the second encoded reconstruction version to determine the abnormal regional image quality association, and to determine the regions with different fidelity levels between the first encoded reconstruction version and the second encoded reconstruction version as image quality association regions. The adjustment module 440 is used to update the encoding protection area range of the corresponding knowledge point according to the image quality associated area, and to adjust the encoding parameters and reconstruction parameters of the region within the encoding protection area range in the unplayed video frame and the review video segment.

[0157] The training video intelligent processing system of this application embodiment is used to implement the aforementioned training video intelligent processing method. Therefore, the specific implementation of the training video intelligent processing system can be found in the embodiment section of the training video intelligent processing method above. The specific implementation can be referred to the description of the corresponding embodiment, which will not be repeated here.

[0158] Electronic devices may include processors and memory storing computer program instructions.

[0159] Specifically, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0160] The memory may include a large-capacity storage device for data or instructions. For example, and not limitingly, the memory may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, the memory is non-volatile solid-state memory.

[0161] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of this disclosure.

[0162] The processor reads and executes computer program instructions stored in memory to implement any of the training video intelligent processing methods in the above embodiments.

[0163] In one example, the electronic device may also include a communication interface and a bus. The processor, memory, and communication interface are connected via the bus and communicate with each other.

[0164] The communication interface is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0165] A bus, including hardware, software, or both, couples components of an online data traffic metering device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0166] The electronic device can execute the training video intelligent processing method in the embodiments of this application, thereby realizing the training video intelligent processing method described in conjunction with the accompanying drawings.

[0167] Furthermore, in conjunction with the intelligent training video processing method in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the intelligent training video processing methods in the above embodiments.

[0168] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0169] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0170] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0171] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0172] The above provides a detailed description of the intelligent processing method and system for training videos provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for intelligent processing of training videos, characterized in that, include: Get training videos and a timeline of key points; The training video is segmented into frames and its content is identified, and the image regions that carry the teaching content within the time period corresponding to the knowledge point timeline are identified as key regions. The outer region of the critical region is defined as the edge region, and the image region adjacent to the edge region but not marked as the critical region is defined as the non-critical region. A first encoding reconstruction version and a second encoding reconstruction version are generated based on the critical region, the edge region, and the non-critical region. The first encoding reconstruction version reduces the fidelity of the edge region and the non-critical region, while the second encoding reconstruction version increases the fidelity of the critical region and the edge region. The fidelity is determined by the encoding parameters and the reconstruction parameters. After playing the corresponding time periods of the first and second encoded reconstructed versions, the number of pauses, replays, and incorrect answers are collected as response data within a preset response time window; By comparing the response data of the first encoded reconstruction version and the second encoded reconstruction version, regional image quality correlation anomalies are determined, and the regions with different fidelity levels between the first encoded reconstruction version and the second encoded reconstruction version are identified as image quality correlation regions. Update the encoding protection area range of the corresponding knowledge point based on the image quality associated area, and adjust the encoding parameters and reconstruction parameters of the region within the encoding protection area range in the unplayed video frame and the review video segment.

2. The method according to claim 1, characterized in that, The teaching content includes blackboard writing, formulas, courseware text, and practical exercises; The step of performing frame segmentation and content recognition on the training video, and identifying the image regions carrying the teaching content within the time period corresponding to the knowledge point timeline as key regions, includes: The training video is segmented into multiple video frames. Text recognition and target detection are performed on the multiple video frames within the time period corresponding to the knowledge point timeline to obtain the image region carrying the teaching content. The image regions whose confidence level is greater than or equal to a preset confidence threshold are identified as the key regions.

3. The method according to claim 1, characterized in that, The definition of using the outer region of the critical region as the edge region and the image region adjacent to the edge region but not marked as the critical region as the non-critical region includes: The minimum bounding rectangle of the key region is determined as the expansion reference. The expansion distance is determined according to the preset video frame size ratio. The expansion reference is expanded according to the expansion distance to obtain the expansion area as the edge region. Image regions whose boundary distance from the edge region is less than or equal to a preset adjacency distance and which are not marked as critical regions are identified as non-critical regions.

4. The method according to claim 1, characterized in that, The encoding parameters include bitrate allocation parameters and quantization parameters, and the reconstruction parameters include edge enhancement parameters and super-resolution reconstruction parameters; The process of generating a first encoded reconstruction version and a second encoded reconstruction version based on the key region, the edge region, and the non-key region includes: Corresponding encoding parameters and reconstruction parameters are configured for the critical region, the edge region, and the non-critical region, and the fidelity level is obtained by mapping the encoding parameters and the reconstruction parameters. Keeping the encoding and reconstruction parameters of the critical regions unchanged, and reducing the fidelity of the edge regions and the non-critical regions, the first encoded reconstruction version is generated; Improve the fidelity of the critical region and the edge region, while keeping the encoding and reconstruction parameters of the non-critical region unchanged, to generate the second encoded reconstruction version.

5. The method according to claim 1, characterized in that, After playing the corresponding time periods of the first and second encoded reconstructed versions, the number of pauses, replays, and incorrect answers are collected as response data within a preset response time window, including: The knowledge point time periods corresponding to the first and second encoding reconstruction versions are determined based on the knowledge point time axis. After the end of the knowledge point period, a preset reaction time window is activated, which is determined based on the duration of the knowledge point period. Within a preset response time window, the number of pauses, replays, and incorrect answers are collected as response data.

6. The method according to claim 5, characterized in that, The step of comparing the response data of the first encoded reconstruction version and the second encoded reconstruction version to determine the regional image quality correlation anomaly includes: Calculate the data difference between the response item data of the same type corresponding to the first encoded reconstruction version and the second encoded reconstruction version; The response item data whose data difference is greater than or equal to the corresponding preset difference threshold is determined as the target response item data, and the preset difference threshold is determined based on the historical average of the corresponding response item data; When the target response item data of the first encoded reconstruction version is greater than the target response item data of the second encoded reconstruction version, the anomaly type is determined to be a regional image quality association anomaly.

7. The method according to claim 6, characterized in that, The method further includes: To obtain the logical relationships between knowledge points; When the target response item data does not exist, the prerequisite knowledge points that have a dependency relationship with the knowledge points corresponding to the first encoding reconstruction version and the second encoding reconstruction version are determined according to the logical association relationship. When the correct answer rate of the prerequisite knowledge points is less than the preset correct answer rate threshold, the anomaly type is determined to be a content comprehension-related anomaly, and the coding protection area range of the knowledge points corresponding to the first and second coding reconstruction versions remains unchanged.

8. The method according to claim 1, characterized in that, The step of updating the encoding protection region range of the corresponding knowledge point based on the image quality association region, and adjusting the encoding parameters and reconstruction parameters of the region within the encoding protection region range in the unplayed video frames and review video segments, includes: When the image quality associated region only includes the critical region, the range of the encoding protection region is updated to cover the critical region. When the image quality associated region includes the edge region but does not include the non-critical region, the range of the encoding protection region is updated to cover the critical region and the edge region. When the image quality associated region includes the non-critical region, the range of the encoding protection region is updated to cover the critical region, the edge region and the non-critical region. Taking the unplayed video frames after the playback progress that caused the image quality correlation anomaly in the region and the review video segments generated according to the corresponding time period of the knowledge point timeline as the adjustment objects, the encoding parameters and reconstruction parameters of the regions within the encoding protection area of ​​the adjustment objects are adjusted respectively, and the fidelity level is improved.

9. A training video intelligent processing system, characterized in that, include: The acquisition module is used to acquire training videos and a timeline of knowledge points; The determination module is used to perform frame segmentation and content recognition on the training video, and to take the image region carrying the teaching content within the time period corresponding to the knowledge point timeline as the key region. The determining module is further configured to use the outer region of the key region as the edge region and the image region adjacent to the edge region but not marked as the key region as the non-key region; A generation module is used to generate a first encoding reconstruction version and a second encoding reconstruction version based on the key region, the edge region, and the non-key region. The first encoding reconstruction version reduces the fidelity level of the edge region and the non-key region, and the second encoding reconstruction version increases the fidelity level of the key region and the edge region. The fidelity level is determined by encoding parameters and reconstruction parameters. The generation module is also used to collect the number of pauses, replays, and incorrect answers as response data within a preset reaction time window after playing the corresponding time periods of the first and second encoded reconstructed versions. The determination module is also used to compare the response item data of the first encoding reconstruction version and the second encoding reconstruction version to determine the regional image quality association anomaly, and to determine the region with different fidelity levels between the first encoding reconstruction version and the second encoding reconstruction version as the image quality association region; The adjustment module is used to update the encoding protection area range of the corresponding knowledge point according to the image quality associated area, and to adjust the encoding parameters and reconstruction parameters of the region within the encoding protection area range in the unplayed video frame and the review video segment.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the training video intelligent processing method as described in any one of claims 1 to 8 when executing the computer program.

Images