Course video key frame intelligent identification method based on AI visual attention mechanism

Abstract

The application relates to the technical field of video recognition, and discloses a course video key frame intelligent recognition method based on an AI visual attention mechanism, which comprises the following steps: acquiring a visual saliency feature map of a video frame and calculating a global attention gravity center coordinate, constructing a spatial second moment tensor by using the visual saliency feature map to determine an anisotropy coefficient, then performing nonlinear weighted processing on the trajectory distribution density in a space-time trajectory space, calculating a second acceleration residual based on the processed trajectory evolution process, and determining a key frame in combination with the trajectory distribution density and the second acceleration residual. The application uses an anisotropy regulation mechanism to suppress non-content dynamic interference, checks the integrity of teaching content generation through the second acceleration residual, solves the problem of lag in semantic turning point capture under a dynamic background, and enhances the semantic density of extracted sequences and the recognition stability.

Description

Technical Field

[0001] This invention relates to an intelligent recognition method for keyframes in course videos based on an AI visual attention mechanism, belonging to the field of video recognition technology. Background Technology

[0002] Currently, in the automated processing of course videos, keyframe extraction is a fundamental step in achieving video content indexing, summary generation, and semantic understanding. Existing technologies typically use methods such as inter-frame pixel difference, optical flow field analysis, or saliency feature statistics to capture visual changes in video frame sequences. By monitoring dramatic fluctuations in visual energy, they identify content turning points and exhibit good stability in general video processing scenarios with static backgrounds and simple foreground targets. However, in real-world teaching applications, course videos are generally constrained by the coexistence of high-dynamic motion noise and low-energy semantic evolution. Teachers frequently move around in front of the projection area or blackboard, accompanied by gestures, generating non-semantic kinetic energy that dominates the visual experience. Meanwhile, the energy fluctuations at the pixel level from page turning or blackboard text updates are far smaller than the interference signals generated by human movement. Existing recognition methods based on the absolute value of visual energy face performance issues in sensitivity setting when dealing with such complex scenarios, namely, it is difficult to maintain the perception accuracy of teaching content updates while suppressing teacher motion noise.

[0003] Besides the image energy distribution recognition strategy, the back-end recognition logic processing strategy also has shortcomings. For example, Chinese invention patent CN118279777A discloses a method and device for identifying video keyframes. It improves processing efficiency by establishing a mapping relationship between video category and traversal step size. The judgment criterion essentially relies on the statistical similarity of global features between frames. In specific scenarios in teaching where the energy level of action noise far exceeds the energy level of content signal, the global similarity calculation is easily affected by the instructor's body displacement, making it difficult to separate non-semantic actions from substantive content at the feature level. The dynamic step size adjustment mechanism lacks steady-state perception capability for randomly occurring limb occlusion or light environment fluctuations, resulting in the extraction of keyframes being mixed with noise. The intermediate transition state affects the integrity of the teaching logic. Specific analysis reveals the following shortcomings of existing technologies: 1. When the sensitivity of visual change detection is increased, the teacher's displacement within the teaching area is easily identified as a semantic turning point, generating a large amount of non-semantic redundant information; 2. When the judgment threshold is increased to suppress human movement interference, it easily leads to missed detection of minor text updates, formula derivations, or local chart changes in the courseware, undermining the integrity of the teaching logic; 3. By introducing target detection algorithms to separate the human body from the background, in complex lighting environments such as the teaching site, not only are the computational resources required of the computing equipment high, but false jumps in the center of gravity trajectory are also generated when limb occlusion or projection flicker occurs.

[0004] Therefore, the technical problem to be solved by this invention is how to achieve topological decoupling between non-semantic action interference and substantive content evolution from the principle level without increasing the hardware resource load, and how to determine the formation of semantic steady state by monitoring the quasi-steady-state evolution law of the visual attention center of gravity in phase space. Summary of the Invention

[0005] To address the problems mentioned in the background art, the technical solution of the present invention is as follows: A method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism, comprising the following steps:

[0006] Step 101: Obtain the visual saliency feature map of each video frame in the video stream to be identified, and calculate the global attention centroid coordinates of the visual saliency feature map.

[0007] Step 102: Calculate the second-order central moment relative to the global attention centroid coordinates using the visual saliency feature map to generate the second-order moment tensor of the saliency space.

[0008] Step 103: Perform eigenvalue decomposition on the second moment tensor of the saliency space, extract the principal eigenvalues ​​and secondary eigenvalues, and calculate the deviation ratio between the principal eigenvalues ​​and secondary eigenvalues ​​to determine the anisotropy coefficient characterizing the spatial distribution orientation of the attention field.

[0009] Step 104: Based on the preset weight mapping rules, the anisotropic coefficients are converted into interference discrimination weights, and the interference discrimination weights are used to nonlinearly weight the trajectory distribution density composed of the global attention centroid coordinates of continuous video frames.

[0010] Step 105: Perform a second-order difference operation on the global attention centroid coordinate sequence after nonlinear weighting to obtain the second-order acceleration residual characterizing the degree of motion change;

[0011] Step 106: When the trajectory distribution density exceeds the preset steady-state threshold and the second-order acceleration residual is lower than the preset pause threshold, the current video frame is determined to be a key frame.

[0012] Preferably, step 104 includes: when the anisotropy coefficient exceeds a preset noise threshold of 0.6, determining that the current saliency energy distribution is dominated by a strip-shaped dynamic interference, and reducing the weight allocation ratio of the global attention centroid coordinates of the frame when calculating the trajectory distribution density; when the anisotropy coefficient is lower than 0.6 and the trace of the second moment tensor of the saliency space shows a continuous decreasing trend, reducing the length of the sliding time window.

[0013] Preferably, in step 104, the process of nonlinearly weighting the trajectory distribution density includes: calculating the local trajectory distribution density using a coordinate sequence consisting of the coordinates of the global attention centroid of 10 to 30 consecutive frames within a sliding time window; and correcting the local trajectory distribution density based on the interference discrimination weight, wherein the interference discrimination weight is negatively correlated with the anisotropy coefficient to suppress the divergent trajectory components generated by the teacher's walking.

[0014] Preferably, step 105 includes: extracting the displacement vector of the global attention centroid coordinates on the time series, and calculating the second derivative of the displacement vector with respect to time to generate a real-time acceleration vector; calculating the difference between the real-time acceleration vector and the preset motion inertia model to obtain the second-order acceleration residual, which is used to determine the stopping state of the blackboard writing action or demonstration operation.

[0015] Preferably, in step 101, the process of calculating the global attention centroid coordinates includes: extracting multi-scale features of the video frame using a convolutional neural network, and calculating the saliency weight of each pixel based on the spatial attention operator; and performing a weighted average of the pixel coordinate values ​​in the video frame according to their corresponding saliency weights to generate the global attention centroid coordinates representing the location of visual attention aggregation.

[0016] Preferably, in step 103, the formula for calculating the anisotropy coefficient A is: ,in, These are the principal eigenvalues ​​of the second-order moment tensor in the saliency space. Let be the second eigenvalue of the second moment tensor of the saliency space, and .

[0017] Preferably, the method further includes the following steps: Step 107, performing regional connectivity analysis on the visual saliency feature map and determining the ambient light and shadow interference region based on the degree of anisotropy of the saliency distribution; reconstructing the global attention centroid coordinates by suppressing the pixel distribution weights of the saliency feature map in the principal axis direction within the interference region.

[0018] Preferably, the method further includes the following steps: Step 108, when the distribution intensity of the visual saliency feature map undergoes a step change within 500ms, the current global attention centroid coordinates are corrected by combining local texture complexity features to compensate for the centroid shift caused by teacher occlusion.

[0019] Preferably, step 106 includes: obtaining the motion stop time when the second-order acceleration residual is lower than a preset pause threshold, and synchronizing the key frame output time with the motion stop time to filter out redundant frames during the teaching pause.

[0020] Preferably, steps 101 to 106 are executed by a video processing system, which includes a feature extraction unit, a tensor analysis unit, and a keyframe determination unit. The feature extraction unit is used to execute step 101 to generate a salient feature map. The tensor analysis unit is used to execute steps 102 and 103 to construct a second-order moment tensor and determine the anisotropy coefficients. The keyframe determination unit is used to execute steps 104 to 106 to identify keyframes.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] 1. In the intelligent recognition of keyframes in course videos, a quasi-steady-state determination mechanism for visual attention in phase space is constructed. The formation process of semantic attractors is captured by using the spread density entropy of the center of gravity trajectory. This changes the traditional logic of determining keyframes based on absolute fluctuations in visual energy. It effectively distinguishes between high-frequency displacement noise generated by teacher actions and topological steady-state components formed by courseware updates. When non-semantic human movement occurs in the video stream, the spread density entropy remains in a high-level fluctuation state. When new teaching content appears and tends to stabilize, the center of gravity of attention quickly converges to the new attractor region in phase space, causing the entropy value to drop sharply. This makes the recognition of keyframes no longer limited by the energy intensity at the pixel level, thus accurately capturing the critical point of semantic transition in complex dynamic backgrounds.

[0023] 2. To improve the signal-to-noise ratio of the center of gravity trajectory under complex working conditions, combined with the anisotropic control mechanism of saliency distribution, morphological topological analysis is performed on the saliency feature map using the second-order central moment tensor. By quantifying the directional characteristics of energy distribution, long strips and diffuse saliency interferences caused by large-scale limb displacement or ambient light flicker are identified and removed. This filtering mechanism based on morphological cohesion weights ensures that the center of attention is always driven by the subject of teaching content with higher geometric density and stronger topological stability, making the evolution trajectory of the center of gravity coordinates in phase space more inclined to the true geometric center of the semantic subject, thus enhancing the recognition stability of the system under high-frequency teacher interaction or light environment fluctuation conditions.

[0024] 3. To ensure the semantic completeness and logical density of keyframe extraction, the second-order acceleration residual of the centroid coordinates is used as a closed-loop gating for semantic generation. The final state of content generation is verified by monitoring the disappearance of micro-motion pulses, which effectively compensates for the logical blind spots that exist when relying solely on topological steady-state judgment. By recognizing the disappearance of physical inertia when writing or operating actions stop, the output time of the keyframe is dynamically aligned with the time when the knowledge point is fully expressed, avoiding invalid extraction caused by temporary pauses in the teaching process or teacher thinking. This results in a keyframe sequence with extremely high semantic density, significantly reducing the computational load of subsequent video structuring processing. Attached Figure Description

[0025] Figure 1 This is a flowchart of the keyframe recognition method of the AI ​​visual attention mechanism of the present invention;

[0026] Figure 2 This is a schematic diagram illustrating the principle and logical architecture of the keyframe intelligent recognition algorithm of this invention. Detailed Implementation

[0027] The method provided by the present invention will be described in detail below with reference to specific embodiments. It should be noted that the following embodiments are intended to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0028] A method for intelligent keyframe recognition in course videos based on AI visual attention mechanisms addresses the challenge of maintaining accurate perception of teaching content updates while suppressing motion noise. This is because teachers frequently move in front of the projection area or blackboard, generating visual kinetic energy far exceeding pixel energy fluctuations caused by page turning or text updates. The present invention addresses this by monitoring the evolution of the visual attention centroid in phase space to establish a quasi-steady-state determination mechanism for identifying substantial semantic turning points. The method utilizes the following logical procedures: acquiring visual saliency feature maps of each video frame in the video stream to be recognized, and calculating the global attention centroid coordinates of the visual saliency feature maps; and using the visual saliency feature maps to calculate the coordinates of the global attention centroid relative to the global attention... The second central moment of the centroid coordinates is used to generate a saliency spatial second moment tensor. Eigenvalue decomposition is performed on this tensor to determine anisotropy coefficients representing the spatial distribution orientation of the attention field. Based on this, the anisotropy coefficients are converted into interference discrimination weights, and the trajectory distribution density formed by the global attention centroid coordinates is nonlinearly weighted. Finally, a second-order difference operation is performed on the weighted coordinate sequence, and keyframes are determined by combining the trajectory distribution density with the obtained second-order acceleration residuals. To address the negative impact of background interference from teacher walking or gestures on semantic centroid localization, the system uses a convolutional neural network to extract multi-scale features from video frames and calculates the saliency weight of each pixel based on the spatial attention operator. The pixel coordinate values ​​in the video frame are weighted and averaged according to their corresponding saliency weights to generate global attention centroid coordinates representing the location of visual attention convergence. These centroid coordinates are calculated using the following formula: , Where X and Y are the horizontal and vertical coordinates of the global attention centroid, respectively, and P(x,y) is the saliency weight of the corresponding pixel (x,y). This procedure reduces the dimensionality of the high-dimensional video image to the centroid trajectory in a two-dimensional space.

[0029] Since limb movements or flickering light and shadow appear as elongated or diffuse distributions in the visual saliency map, this noise can cause deviations in the centroid coordinates. To eliminate this physical interference, the system uses the visual saliency feature map to calculate the second-order central moment relative to the global attention centroid coordinates, generating a second-order moment tensor in the saliency space, which forms the covariance matrix M. Eigenvalue decomposition is then performed on matrix M to extract the principal eigenvalues. and secondary eigenvalues And calculate the anisotropy coefficient A: Where A is the anisotropy coefficient. These are the principal eigenvalues ​​of the second-order moment tensor in the saliency space. Let be the second eigenvalue of this tensor, and satisfy . To address trajectory deviations induced by significant teacher movement, the system converts the anisotropy coefficient A into interference discrimination weights based on a preset weight mapping rule, thereby correcting the trajectory distribution density. In specific applications, the local trajectory distribution density is calculated using a coordinate sequence consisting of 10 to 30 consecutive frames of global attention centroid coordinates within a sliding time window. When the anisotropy coefficient A exceeds a preset noise threshold of 0.6, it is determined that the current distribution is dominated by elongated dynamic interference. This judgment logic is based on the consistency of the physical projection of the significant energy field along its major axis. Because the pixel momentum fluctuations generated by the teacher's movement have significant directional orientation, the dominant feature value... Much larger than the secondary eigenvalue Substantive courseware content is typically distributed in isotropic, dense clusters. In practice, this is achieved through a nonlinear mapping function. The system maps interference discrimination weights in real time, physically reducing the displacement signal when the anisotropy coefficient exceeds a threshold during trajectory distribution accumulation. This reduces the weight allocation of the centroid coordinates in calculating the trajectory distribution density for that frame. If the anisotropy coefficient A is below 0.6 and the trace of the second-order moment tensor in the saliency space shows a continuous decreasing trend, the system reduces the sliding time window length to improve the sensitivity to the formation of new attractors, thus adjusting the interference discrimination weights relative to the anisotropy coefficient. It shows a negative correlation, suppresses the divergent components generated by non-content-based displacement, and enhances the stability of the center of gravity trajectory under teacher interaction conditions. The noise threshold of 0.6 is determined by statistically analyzing the distribution of video databases containing the displacement characteristics of lecturers and extracting the anisotropy coefficient at the 90th cumulative percentile under the human body long axis motion condition. The pause threshold of 0.05 is determined by collecting background noise signals from the teaching site, calculating the root mean square error of the second derivative of the center of gravity displacement vector when the action input is missing, and using 3 times the root mean square error as the action stop judgment index to suppress the interference of lecturer displacement and verify the generation of the final state of blackboard writing or demonstration operation.

[0030] To filter out invalid keyframes caused by temporary pauses in teaching or teacher reflection, the system extracts the displacement vector of the global attention centroid coordinates over time and calculates its second derivative with respect to time to generate a real-time acceleration vector. The system then calculates the difference between the real-time acceleration vector and a preset motion inertia model. This preset model uses a Kalman filter algorithm to linearly predict the motion trend of the preceding 15 frames. Because the actual displacement vector of the physical gesture at the moment the writing action ends will deviate from the predictor's prediction envelope, it induces a step contraction in the second-order difference result, thus obtaining a second-order acceleration residual. This residual is used to determine the stopping state of the writing action or demonstration operation, reflecting the physical inertial state of the writing or demonstration action. Finally, the system monitors the trajectory distribution density and the second-order acceleration residual. When the trajectory distribution density exceeds a preset steady-state threshold and the second-order acceleration residual is lower than a preset pause threshold, the current video frame is determined to be a keyframe, and the system obtains... The motion stopping moment is selected when the second-order acceleration residual is lower than the preset pause threshold, and the keyframe output moment is synchronized with the motion stopping moment to ensure that the output keyframe is at the critical point of complete knowledge expression. This realizes the closed-loop association between semantic transition capture and physical action stopping. The system uses a point set composed of continuous coordinate sequences within the sliding time window to determine the trajectory distribution density D. Specifically, the system iterates through the Euclidean distance of each global attention centroid coordinate relative to the current centroid position within the window, and performs mapping processing on each distance component using a kernel density estimation function. This kernel density estimation function uses a distance reciprocal operator with a bandwidth constant of 20 pixels. Each Euclidean distance pixel value is added with an offset of 1 to prevent division by zero and the reciprocal is taken. Then, all reciprocal image values ​​within the sliding time window are arithmetically averaged. The arithmetic mean of the obtained image values ​​is defined as the aggregation index characterizing the compactness of the attention distribution, thus determining the physical evolution path of the coordinate point cloud converging towards the semantic attractor.

[0031] Example 1: In a specific lecture hall scenario, the video capture resolution is set to 1920×1080 pixels. The environment experiences brightness fluctuations caused by external light sources. The lecturer frequently moves laterally between the projection screen and the blackboard at a speed of 1.5 m / s. Simultaneously, the projection screen periodically refreshes the teaching materials. Because the lecturer's body features exhibit high-energy dynamic components in the visual saliency field, and the long axis of the saliency distribution closely matches the walking path, the X and Y coordinates of the global attention centroid shift. The system obtains the visual saliency feature map of the video frame through the feature extraction unit and calculates the real-time global attention centroid coordinates using the following formula: , In this system, X and Y are the x and y coordinates of the global attention centroid, respectively, and P(x,y) is the saliency weight of the corresponding pixel (x,y). This coordinate trajectory initially contains a large number of non-semantic components generated by the instructor's movement. To identify and suppress the long-axis dynamic noise caused by the instructor's displacement, the system performs morphological topology analysis using the second-order moment tensor of the saliency space. By calculating the second-order central moment of the visual saliency feature map relative to the global attention centroid coordinates, a covariance matrix M representing the energy distribution characteristics is generated. Eigenvalue decomposition is then performed on this matrix M to extract the principal eigenvalues. and secondary eigenvalues The anisotropy coefficient is calculated according to the following formula. : Where A is the anisotropy coefficient. Principal eigenvalues, It is a secondary eigenvalue and satisfies Under this condition, the salient field formed by the instructor's walking exhibits directional characteristics, making... Much larger The calculated anisotropy coefficient A reaches 0.75, exceeding the preset noise threshold of 0.6. According to the preset weight mapping rule, the anisotropy coefficient A is converted into a lower interference discrimination weight, reducing the contribution of the current video frame in calculating the trajectory distribution density. This achieves the removal of interference from the instructor's actions at the logical level, allowing the system to focus on the densely distributed text area of ​​the courseware.

[0032] When the instructor stops moving and begins the demonstration, the saliency energy converges in the phase space towards the region containing the teaching content, causing the trace of the second-order moment tensor in the saliency space to continuously decrease, and the anisotropy coefficient A to drop to around 0.35. The system recognizes this physical characteristic of decreasing anisotropy, determines that semantic steady state is forming, and simultaneously reduces the sliding time window length to improve the sensitivity to capturing the details of the updated courseware. At this time, the global attention centroid coordinates collected by the feature extraction unit show a high degree of spatial aggregation, and the calculated trajectory distribution density climbs from a low value to above the preset steady state threshold. The keyframe determination unit extracts the second derivative of the displacement vector between consecutive frames and calculates the second-order acceleration residual, which represents the sudden change in motion state. When the second-order acceleration residual is detected to be continuously below the pause threshold of 0.05, and the trajectory distribution density remains at the preset steady state threshold, the system stops moving. When the state threshold is above, the system determines that the current video frame has completed the delivery of the teaching content for this stage. In the final keyframe confirmation stage, the system obtains the motion stop time when the second-order acceleration residual is lower than the preset pause threshold, and synchronizes the keyframe output time with this time. Through this synchronization calibration, the system marks the video frames with complete text derivation and suppressed limb interference in the dynamic background of the instructor's movement as keyframes. By using the anisotropic control mechanism to quantify the saliency distribution directionality, combined with the monitoring of physical inertial stopping characteristics by the second-order acceleration residual, the system achieves a keyframe recognition accuracy of no less than 98% without increasing the hardware computing load. Through the above procedures, the quasi-steady-state determination of visual attention is combined with the motion pulse disappearance characteristics to ensure that the extracted keyframe sequence completely restores the teaching logic.

[0033] Example 2: To verify the stability of this method in capturing keyframes under complex dynamic conditions, the experiment extracted a teaching resource library containing 100 sets of classroom videos as input data. The video acquisition resolution was set to 1920×1080 pixels, and the frame rate was 30Hz. The video stream contained interference features such as instructors walking, obscuring the blackboard, and light and shadow fluctuations. To simulate the electromagnetic environment, Gaussian white noise with a signal-to-noise ratio of 25dB was superimposed on the video stream. The experimental platform included a processor with floating-point arithmetic capabilities, and the sampling frequency was set to 30Hz. During parameter calibration, the sliding time... The selection of the window length k is constrained by the real-time capture performance and statistical stability. If k is too large, it will lead to a delay in the semantic inflection point localization; if k is too small, it will amplify the coordinate jitter caused by pixel fluctuations. Based on the physical momentum distribution law of the teaching action, the initial value range of k is determined to be 10 to 30 frames, and dynamic correction is performed according to anisotropic features during the calculation process. The experiment establishes a control system to verify the synergistic effect between various technical features. The sample group of this invention adopts a scheme that includes anisotropic control and second-order acceleration residual verification. The control group A removes the anisotropic coefficients. In the guided interference discrimination weight allocation process, control group B adopted the traditional judgment method based on the absolute fluctuation of visual energy. Under the condition that the lecturer moved laterally at a speed of 1.5 m / s, the original saliency energy distribution obtained by monitoring showed a directional pattern. The experimental data showed that control group B could not isolate the pixel momentum fluctuations caused by human displacement, and generated 15 redundant keyframes during the displacement, resulting in a decrease in recognition accuracy to 72.4%. Although control group A suppressed some random jitter through trajectory density analysis, it did not perform weight compensation for the long strip dynamic noise, and its global attention centroid coordinates generated a root mean square error of 15.4 pixels in the walking direction, resulting in 6 intermediate frames in the extracted keyframes from the blackboard writing process. In contrast, the sample group of this invention used the visual saliency feature map to calculate the second-order central moment relative to the global attention centroid coordinates, generated a saliency space second-order moment tensor representing the energy distribution characteristics, and performed eigenvalue decomposition to extract the principal eigenvalues. and secondary eigenvalues According to the formula The anisotropy coefficient A is calculated, where A is the anisotropy coefficient. Principal eigenvalues, It is a secondary eigenvalue and satisfies Actual test data shows that when A reaches 0.78, the system reduces the weight of the centroid coordinates in the trajectory density calculation to 0.12, which reduces the root mean square error of the centroid trajectory from 15.4 pixels in the control group A to 2.3 pixels, thereby enhancing the ability of the centroid trajectory to lock onto the main body of the teaching content.

[0034] When verifying the rationality of the parameter boundaries, gradient tests were conducted against the preset noise threshold. This threshold was set to 0.2, 0.6, and 0.9 respectively. The test results showed that when the threshold was set to 0.9, the system could not detect the morphological topological changes caused by the instructor's limb displacement, leading to a 45.2% increase in redundant frame rate. When the threshold was set to 0.2, normal courseware chart updates were misjudged as dynamic noise due to local saliency distribution characteristics, resulting in a 36.2% false negative rate. However, when the threshold was set to 0.6, the system exhibited stable discrimination sensitivity. Combined with the monitoring of second-order acceleration residuals, this allowed the system to locate the physical critical point for the complete expression of knowledge points. The keyframe determination unit extracted the displacement vector of the global attention centroid coordinates on the time series, calculated its second derivative with respect to time to determine the real-time acceleration vector, and determined the second-order acceleration residual by calculating the difference between this real-time acceleration vector and the preset motion inertia model. The measured data showed that within 125.6 ms after the writing action stopped, the second-order acceleration residual decreased from 2.45 m / s². 2 It dropped to 0.032 m / s 2The pause threshold is consistently below 0.05, thereby aligning the keyframe output time with the motion stop time to solve the problem of semantic capture lag in dynamic backgrounds. Under the condition of introducing Gaussian noise and limb displacement noise, the keyframe recognition accuracy of the sample group of this invention reaches 98.2%, the recall rate reaches 97.6%, and the single frame processing time is maintained at 15.2ms. This result confirms that monitoring the quasi-steady-state evolution law of the visual attention center of gravity in phase space is effective for keyframe recognition. The entire logical procedure achieves topological decoupling between non-semantic action interference and content evolution without relying on additional target detection algorithms.

[0035] Example 3: In a pre-defined thematic teaching simulation environment, the video acquisition system uses a processor with floating-point arithmetic capabilities to process real-time video streams. The video resolution is set to 1920×1080 pixels, and the sampling frequency is set to 30Hz. When the instructor performs a chemical demonstration, their limbs move within the teaching area. Due to light and shadow occlusion and pixel energy fluctuations caused by the texture of clothing, the saliency feature map exhibits anisotropic characteristics. The following determined logical steps are executed to calculate the second-order acceleration residual: Obtain the global attention centroid coordinate sequence at three consecutive video frame times t, t-1, and t-2, denoted as... , and ; Calculate the displacement vector between adjacent frames, the displacement between the first frame Displacement between the second and third frames Perform a first-order difference operation on the displacement vector to determine the real-time acceleration vector. Δt is taken as 33.3ms; finally, the real-time acceleration vector is subtracted from the mean motion inertia model calculated based on the first 50 frames of motion sequence, and the Euclidean norm of the result is defined as the second-order acceleration residual.

[0036] To eliminate interference in the mapping relationship of the discrimination weights, the system uses a defined transfer function to map the anisotropy coefficient A to a weight value w. The specific procedure is as follows: when the anisotropy coefficient A is in the range of 0 to 0.6, the weight value w is constantly set to 1.0, indicating that the significance distribution is in a dense steady state; when the anisotropy coefficient A is in the range of 0 to 0.6, the weight value w is .... When the value exceeds 0.6 but is not higher than 1.0, the system initiates nonlinear suppression logic. According to the formula w = 1 - (A - 0.6) / 0.4, the interference discrimination weight of the current frame is calculated. In a set of measured data, when the instructor rapidly waves their arm, causing A to jump to 0.85, the calculated weight value w automatically shrinks to 0.375. This weight acts on the accumulation process of the trajectory distribution density, reducing the false trajectory energy generated by limb movements by more than 62.5%, thereby achieving in-situ physical calibration of the semantic center of gravity and preventing false triggering of keyframes induced by non-content-related dynamic components. During nonlinear weighting, the system uses the interference discrimination weight w to correct the response amplitude of the kernel density estimation function. When the anisotropy coefficient A is between 0.6 and 1.0, the gain coefficient of the corresponding frame in the trajectory accumulation sequence is reduced to decrease the impact of divergent components generated by limb displacement on local aggregation indicators. This procedure ensures that the weighted trajectory distribution density only reflects... The system reflects the topological steady-state characteristics generated by the evolution of teaching content, achieving logical separation between semantic turning points and dynamic background interference. Regarding the determination of the preset pause threshold of 0.05, the system executes a standardized calibration procedure before the recognition process begins: 300 frames of background noise signals are collected in a static state of the instructor; the standard deviation of the second derivative of the displacement vector in this state is calculated; and three times this standard deviation is set as the critical point for the pause threshold in the current environment. At the end of the demonstration operation, the second-order acceleration residual drops from 1.25 to 0.028 within 5 consecutive frames, remaining below the calibrated pause threshold of 0.05. Based on this, the system determines that the physical action has entered the closed-loop termination state and confirms this moment as the keyframe output moment. This procedure solves the problem of false overshoot of the second-order acceleration residual caused by sampling jitter, anchoring the marked action of the keyframe to the inflection point of the teaching content generation, improving the semantic integrity of the extracted sequence, and verifying the engineering stability of this solution under complex interactive conditions.

[0037] Example 4: In a specific tiered classroom deployment environment, the system performs a background baseline calibration procedure before the formal recognition process begins. This is to reduce static interference from fixed facilities and ambient lighting on the visual attention center. The system continuously acquires 150 frames of video images in a classroom without any instructors present, and uses the feature extraction unit to calculate the static saliency weight distribution of the scene. The obtained weight distribution is stored in memory as a zero-biased benchmark. During real-time recognition, the feature extraction unit compares the saliency weight P(x,y) generated in the current video frame with the static saliency weight distribution. Perform differential cancellation to generate a dynamic saliency weight map for calculating the coordinates of the global attention centroid.

[0038] To construct the motion inertial model when a new teaching recording task begins, the system executes a pre-processing data filling and inertial baseline initialization procedure to provide a physical reference for calculating the second-order acceleration residual. Within the first 10 seconds after detecting the instructor entering the area, the system continuously records the displacement vector of the global attention center of gravity coordinates at a sampling frequency of 30Hz and calculates the sliding arithmetic mean of the displacement vectors during this period, using it as the initial state variable of the motion inertial model. During the real-time recognition phase, the system synchronously extracts the arithmetic mean of the displacement of the centroid coordinates of the most recent 50 frames within the current sliding time window, and uses it as the current motion component to perform a subtraction operation with the aforementioned 10-second initial state variable. The Euclidean norm of the result is defined as the second-order acceleration residual, thereby ensuring the consistency of the physical inertia judgment benchmark. This procedure establishes a statistical benchmark for the motion envelope at the beginning of the task, reduces the overshoot of the initial value of the second-order acceleration residual caused by the lack of historical data, and maintains the logical association of key frame marker actions in the video stream.

[0039] Example 5: In the offline model construction process, the system extracts features from the training set of labeled video frames to construct a deep residual network for visual saliency recognition. This network comprises four residual stages, each utilizing a skip connection mechanism to transfer local texture features and semantic components. During training, a training dataset containing 5000 labeled keyframes is used. The weight parameters are iteratively updated by minimizing the cross-entropy loss function between the predicted saliency distribution and the manually labeled distribution, ensuring the network can accurately identify the main subject of the teaching content. In the network output layer, the system uses a 7×7 spatial filter to perform convolution operations on multi-scale feature maps and normalizes the spatial dimension of the results using the following formula: Where P(x,y) is the saliency weight at pixel position (x,y), and F(x,y) is the score after performing convolution operation at that position; this procedure determines the spatial mapping relationship of the saliency weights.

[0040] To address dynamic interference from background noise, the system executes an adaptive calibration procedure during the identification process to evaluate the stability of the second-order moment tensor of the saliency space under changing background environment. The system monitors the variance fluctuation of the anisotropy coefficient A within a preset time window. When the variance exceeds the fluctuation limit of 0.15, it is determined that there is a disturbance caused by background light and shadow in the current environment. The system then adjusts the sliding time window length k to 45 frames until the fluctuation of the anisotropy coefficient A returns to below 0.05. This procedure maintains the stability of the center of gravity trajectory under interference conditions by adjusting the response depth of the anisotropy control mechanism.

[0041] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism, characterized in that, Includes the following steps: Step 101: Obtain the visual saliency feature map of each video frame in the video stream to be identified, and calculate the global attention centroid coordinates of the visual saliency feature map. Step 102: Calculate the second-order central moment relative to the global attention centroid coordinates using the visual saliency feature map to generate the second-order moment tensor of the saliency space. Step 103: Perform eigenvalue decomposition on the second moment tensor of the saliency space, extract the principal eigenvalues ​​and secondary eigenvalues, and calculate the deviation ratio between the principal eigenvalues ​​and secondary eigenvalues ​​to determine the anisotropy coefficient characterizing the spatial distribution orientation of the attention field. Step 104: Based on the preset weight mapping rules, the anisotropic coefficients are converted into interference discrimination weights, and the interference discrimination weights are used to nonlinearly weight the trajectory distribution density composed of the global attention centroid coordinates of continuous video frames. Step 105: Perform a second-order difference operation on the global attention centroid coordinate sequence after nonlinear weighting to obtain the second-order acceleration residual characterizing the degree of motion change; Step 106: When the trajectory distribution density exceeds the preset steady-state threshold and the second-order acceleration residual is lower than the preset pause threshold, the current video frame is determined to be a key frame.

2. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, Step 104 includes: when the anisotropy coefficient exceeds a preset noise threshold of 0.6, determining that the current saliency energy distribution is dominated by a strip-shaped dynamic interference, and reducing the weight allocation ratio of the global attention centroid coordinates of the frame when calculating the trajectory distribution density; when the anisotropy coefficient is below 0.6 and the trace of the second moment tensor of the saliency space shows a continuous decreasing trend, reducing the length of the sliding time window.

3. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, In step 104, the process of nonlinearly weighting the trajectory distribution density includes: calculating the local trajectory distribution density using a coordinate sequence consisting of the coordinates of the global attention centroid of 10 to 30 consecutive frames within a sliding time window; and correcting the local trajectory distribution density based on the interference discrimination weight, wherein the interference discrimination weight is negatively correlated with the anisotropy coefficient to suppress the divergent trajectory components generated by the teacher's walking.

4. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, Step 105 includes: extracting the displacement vector of the global attention centroid coordinates on the time series, and calculating the second derivative of the displacement vector with respect to time to generate a real-time acceleration vector; calculating the difference between the real-time acceleration vector and the preset motion inertia model to obtain the second-order acceleration residual, which is used to determine the stopping state of the blackboard writing action or demonstration operation.

5. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, In step 101, the process of calculating the global attention centroid coordinates includes: extracting multi-scale features of video frames using a convolutional neural network, and calculating the saliency weight of each pixel based on the spatial attention operator; and performing a weighted average of the pixel coordinate values ​​in the video frame according to their corresponding saliency weights to generate the global attention centroid coordinates representing the location of visual attention clustering.

6. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, In step 103, the formula for calculating the anisotropy coefficient A is: ,in, These are the principal eigenvalues ​​of the second-order moment tensor in the saliency space. Let be the second eigenvalue of the second moment tensor of the saliency space, and .

7. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, It also includes the following steps: Step 107: Perform region connectivity analysis on the visual saliency feature map and determine the ambient light and shadow interference region based on the degree of anisotropy of the saliency distribution; reconstruct the global attention centroid coordinates by suppressing the pixel distribution weights of the saliency feature map in the principal axis direction within the interference region.

8. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, It also includes the following steps: Step 108: When the distribution intensity of the visual saliency feature map undergoes a step change within 500ms, the coordinates of the current global attention centroid are corrected by combining the local texture complexity features.

9. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, Step 106 includes: obtaining the motion stopping time when the second-order acceleration residual is lower than a preset pause threshold, and synchronizing the keyframe output time with the motion stopping time.

10. The method for intelligent recognition of keyframes in course videos based on AI visual attention mechanism according to claim 1, characterized in that, Steps 101 to 106 are executed by a video processing system, which includes a feature extraction unit, a tensor analysis unit, and a keyframe determination unit. The feature extraction unit is used to execute step 101 to generate a salient feature map. The tensor analysis unit is used to execute steps 102 and 103 to construct a second-order moment tensor and determine the anisotropy coefficients. The keyframe determination unit is used to execute steps 104 to 106 to identify keyframes.

Images