Image device control and video stream discretization processing apparatus based on frame change detection

Abstract

The present application relates to the technical field of image processing and device control, and discloses an image device control and video stream discretization processing device based on frame change detection, which comprises an audio processing module that writes a synchronization pulse mark into continuous and unreduced audio data, an image acquisition and analysis module that acquires a video stream to be processed, extracts a current frame, an adjacent frame and an ambient light compensation value, a frame change detection module that calculates a picture change amplitude, a hardware control module that compares a threshold value, generates a picture state indication and a hardware scheduling instruction, a storage control module that extracts discrete visual data according to the picture state indication, a timestamp binding module that fine-tunes and corrects the time nodes of the discrete visual data by taking the synchronization pulse mark as an absolute anchor point, and generates a lightweight discretized video file. The present application realizes hardware energy efficiency closed-loop regulation and control and data discretization storage by using frame change detection, reduces power consumption and storage occupation, and ensures accurate audio and picture synchronization by using a synchronization pulse mark.

Description

Technical Field

[0001] This invention relates to the field of image processing and device control technology, specifically to an image device control and video stream discretization processing device based on frame change detection. Background Technology

[0002] In teaching recording or demonstration scenarios, traditional image acquisition devices typically use a fixed frame rate to continuously capture live footage throughout the entire timeframe. Because the changes in these scenes exhibit significant uneven temporal distribution, with the image remaining static for most of the time, or only accompanied by localized cursor movement and handwriting, the fixed sampling mechanism generates a large number of repetitive video frames. This processing method not only increases the burden on storage space but also keeps the image acquisition device and its processing circuitry constantly under high load, resulting in high power consumption and reduced battery life.

[0003] Current optimization methods largely rely on manual editing in post-production, making it difficult to achieve real-time hardware energy efficiency management at the data generation source. While some deep learning-based recognition algorithms can identify changes in the image, they are highly dependent on hardware computing power, increasing the cost of portable devices. Furthermore, existing processing solutions, when decomposing video into discrete data, often lack temporal mapping logic between the audio stream and discontinuous visual components, easily leading to audio-visual misalignment or temporal alignment issues in variable frame rate or discrete sampling environments. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an image device control and video stream discretization processing device based on frame change detection. This solves the problems in existing technologies, such as data redundancy and high device power consumption due to fixed sampling mechanisms, and the lack of a discretization decomposition mechanism that takes into account low computing power costs and a matching audio-visual synchronization alignment method.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] This invention provides an image device control and video stream discretization processing device based on frame change detection, comprising an audio processing module, an image acquisition and analysis module, a frame change detection module, a hardware control module, a storage control module, and a timestamp binding module.

[0007] The audio processing module acquires the raw audio, treats the raw audio as continuous and unedited audio data, writes hardware-level synchronization pulse markers into the continuous and unedited audio data, and generates audio data with synchronization pulse markers.

[0008] The image acquisition and analysis module acquires the video stream to be processed and parses it into video frames. It then extracts the current frame and adjacent frames from the video frames to obtain the ambient light compensation value.

[0009] The frame change detection module calculates the magnitude of image change based on the current frame, adjacent frames, and ambient light compensation values, using the illumination compensation pixel difference formula and the binary edge contour comparison formula.

[0010] The hardware control module compares the magnitude of screen changes with preset static thresholds, local thresholds, and global thresholds to generate screen status indicators and hardware scheduling instructions.

[0011] The storage control module extracts discrete visual data based on the screen status indication and records the time nodes of the discrete visual data.

[0012] The timestamp binding module uses the synchronization pulse marker in the audio data with synchronization pulse marker as the absolute anchor point to force fine-tune and correct the time nodes of discrete visual data, generating a lightweight discrete video file.

[0013] In the specific operating mechanism, the audio processing module activates the sound pickup device to collect the sound source on site or uses the audio splitter to strip the original audio track as the original audio. It does not perform filtering, silence detection, trimming or segmentation operations based on signal amplitude on the original audio. It retains the data as is as continuous and unedited audio data and writes it into the hardware-level synchronization pulse marker according to the real-time clock at a pre-set fixed period.

[0014] The image acquisition and parsing module receives instructions from the hardware control module and accordingly acquires the video stream to be processed in either a high-frequency acquisition state or a low-frequency, low-power state by changing the sampling interval of the image acquisition device, increasing the number of blanking periods, or performing frame skipping sampling at the hardware link layer. After parsing the video frames, it retains the unprocessed current frame and performs grayscale processing on the current frame and adjacent frames, calculates the brightness difference to obtain the global average brightness difference, and sets it as the ambient light compensation value.

[0015] The frame change detection module uses the illumination compensation pixel difference formula to calculate the difference between the corresponding pixel brightness values ​​of the current frame and the adjacent frames after normalization and deducting the ambient light compensation value, thus obtaining the global pixel change degree after eliminating illumination interference; at the same time, it extracts the edge feature map, uses the binary edge contour comparison formula to calculate the difference to obtain the edge structure change amount, and performs linear weighting by assigning preset fusion weights to calculate the image change amplitude.

[0016] The hardware control module performs a progressive comparison of the above-mentioned image change amplitude with static thresholds, local thresholds, and global thresholds.

[0017] If the change is less than the static threshold, it indicates static pause and generates a command to reduce the acquisition frame rate; if it is between the static threshold and the local threshold, it indicates local cursor movement and generates a command to reduce the acquisition frame rate; if it is between the local threshold and the global threshold, it indicates handwriting change and generates a command to reduce the acquisition frame rate; if it is greater than or equal to the global threshold, it indicates significant change and generates a command to restore the normal frame rate.

[0018] The storage control module receives screen status indications and performs differentiated extraction. When the screen is static, only a single frame is captured; when the cursor moves locally, the cursor position is extracted to generate a cursor trajectory sequence; when the handwriting changes, handwriting points are extracted to generate a handwritten annotation sequence; when there are significant changes, a baseline continuous video stream is captured to generate a micro-video; combined with the inserted material segments generated by the device, this is collectively referred to as discrete visual data. Finally, the timestamp binding module maps the time nodes of the discrete visual data to the time axis of the audio data, retrieves synchronization pulse markers, and uses these markers as absolute anchor points for forced fine-tuning correction when time offsets occur. It then uses a multimedia encapsulation format to package and encapsulate each track, outputting a lightweight discrete video file and hardware scheduling instructions.

[0019] This invention provides an image device control and video stream discretization processing apparatus based on frame change detection. It has the following beneficial effects:

[0020] 1. This invention performs inter-frame pixel difference comparison and edge feature comparison through a frame change detection module. This eliminates the need for complex neural network algorithms, reducing the device's reliance on computing power and hardware deployment costs, ensuring device stability, and allowing for flexible adaptation to various portable shooting devices, computing terminals, and servers with video processing capabilities. Furthermore, this data extraction method covers five types of discrete visual data across all lecture and recording scenarios. While ensuring the completeness and accuracy of effective visual information, it eliminates redundant static background video frames, reducing the final data volume.

[0021] 2. This invention utilizes a hardware control module to generate hardware scheduling instructions based on the magnitude of image changes. In real-time shooting scenarios, it implements hardware-level low-power control over the front-end image acquisition hardware, reducing ineffective power consumption when the image is static or undergoes minor changes, thus extending the battery life of the shooting device. Furthermore, this device balances real-time shooting control with offline discretization processing of existing video. It can be used in conjunction with image acquisition terminals equipped with image geometric correction capabilities to construct a complete link from image correction to data discretization storage, or it can be implemented independently, offering flexible application scenarios and clear protection boundaries.

[0022] 3. This invention utilizes the audio processing module to retain continuous and unedited audio data as the device's global time reference, and establishes a global time index through periodically written hardware-level synchronization pulse markers. The timestamp binding module uses the pulse markers of the entire audio as absolute anchor points to perform time-series mapping and forced fine-tuning correction on multi-dimensional discrete visual data, achieving precise binding of various types of visual data on the time axis, eliminating asynchronous interference caused by front-end hardware frequency reduction, ensuring that various data tracks are without misalignment or disconnection, and accurately adapting to the subsequent structured synchronous playback requirements of lightweight discrete video files. Attached Figure Description

[0023] Figure 1 This is an architectural diagram of the image device control and video stream discretization processing device based on frame change detection of the present invention;

[0024] Figure 2 This is a flowchart of the image device control and video stream discretization processing device based on frame change detection of the present invention.

[0025] Figure 3 This is a flowchart of the closed-loop control and state determination of the hardware control module of the present invention.

[0026] Figure 4 This is a comparison chart showing the relationship between screen change amplitude and storage rate in an application embodiment of the present invention. Detailed Implementation

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see the appendix Figure 1 and attached Figure 2 This invention provides an image device control and video stream discretization processing apparatus based on frame change detection, including an audio processing module, an image acquisition and analysis module, a frame change detection module, a hardware control module, a storage control module, and a timestamp binding module.

[0029] The audio processing module acquires the raw audio. In real-time shooting scenarios, the audio processing module continuously collects sound from the source using its microphone, which serves as the raw audio. In offline processing scenarios, the audio processing module uses an audio separator to extract the original audio track from pre-stored video data, which serves as the raw audio. The audio processing module does not edit the raw audio, retaining the data as continuous, unedited audio data. Based on the underlying real-time clock, the audio processing module writes hardware-level synchronization pulse markers to the continuous, unedited audio data at fixed intervals set internally by the device, generating audio data with synchronization pulse markers.

[0030] The image acquisition and analysis module receives initial frame rate commands, commands to reduce the acquisition frame rate, or commands to restore the normal frame rate from the hardware control module, and switches its operating state according to the received commands. When it receives an initial frame rate command or a command to restore the normal frame rate, the image acquisition and analysis module enters a high-frequency acquisition state and acquires the video stream to be processed in a high-frequency acquisition state. When it receives a command to reduce the acquisition frame rate, the image acquisition and analysis module enters a low-frequency, low-power state and acquires the video stream to be processed in a low-frequency, low-power state. The video stream to be processed includes a continuous video stream acquired in real time by the image acquisition device or a stock video read sequentially. The image acquisition and analysis module uses a video parser to parse the video stream to be processed into a base continuous video stream and video frames arranged in a time sequence.

[0031] The image acquisition and analysis module extracts the current frame and adjacent frames from the video frames, directly retaining the current frame as the unprocessed current frame. The module then performs grayscale processing on the current frame and adjacent frames, generating grayscale-processed current frames and grayscale-processed adjacent frames. Finally, the module calculates the brightness difference between the grayscale-processed current frame and the grayscale-processed adjacent frames to obtain the global average brightness difference, which is then set as the ambient light compensation value.

[0032] The frame change detection module receives the current frame after grayscale processing, adjacent frames after grayscale processing, and ambient light compensation values. It calculates the global pixel change after removing illumination interference using the illumination compensation pixel difference formula. The module then extracts edge feature maps of the current frame and adjacent frames after grayscale processing using an edge extraction operator. Finally, it calculates the edge structure change in the edge feature maps using a binary edge contour comparison formula. The module assigns preset fusion weights to the global pixel change and edge structure change after removing illumination interference and performs a linear weighting to calculate the magnitude of the image change in the current frame.

[0033] The hardware control module receives the amplitude of the image change and compares it with the internally preset static threshold, local threshold and global threshold.

[0034] If the comparison shows that the change in the image is less than the static threshold, the hardware control module indicates the current image state as static and generates a command to reduce the acquisition frame rate. If the comparison shows that the change in the image is greater than or equal to the static threshold but less than the local threshold, the hardware control module indicates the current image state as local cursor movement and generates a command to reduce the acquisition frame rate. If the comparison shows that the change in the image is greater than or equal to the local threshold but less than the global threshold, the hardware control module indicates the current image state as handwriting change and generates a command to reduce the acquisition frame rate. If the comparison shows that the change in the image is greater than or equal to the global threshold, the hardware control module indicates the current image state as significant change and generates a command to restore the normal frame rate. Static pause, local cursor movement, handwriting change, and significant change are collectively referred to as image state indications.

[0035] The storage control module receives screen status indicators, unprocessed current frames, and reference continuous video streams.

[0036] In the storage processing flow, when the screen status indicator is static, the storage control module captures and saves a single static frame from the unprocessed current frame. When the screen status indicator is partial cursor movement, the storage control module extracts and records the cursor position from the unprocessed current frame, generating a cursor trajectory sequence. When the screen status indicator is handwriting change, the storage control module extracts handwriting points from the unprocessed current frame, generating a handwritten annotation sequence. When the screen status indicator is significant change, the storage control module completely captures the reference continuous video stream from the high-frequency acquisition state, generating a micro-video.

[0037] If a demonstration screen from an image device is detected, the storage control module extracts and generates interstitial material clips. Single-frame static images, cursor trajectory sequences, handwritten annotation sequences, micro-videos, and interstitial material clips are collectively referred to as discrete visual data. The storage control module records the time nodes of the discrete visual data.

[0038] The timestamp binding module receives audio data with synchronization pulse markers, discrete visual data, and corresponding time nodes, and maps the time nodes of the discrete visual data to the time axis of the audio data with synchronization pulse markers. During the mapping process, the timestamp binding module retrieves the synchronization pulse markers from the audio data with synchronization pulse markers.

[0039] The timestamp binding module maps the time nodes of discrete visual data to the timeline of audio data with synchronization pulse markers. The timestamp binding module retrieves synchronization pulse markers from the audio data with synchronization pulse markers.

[0040] If a time offset is found between the time node of the discrete visual data and the nearest synchronization pulse mark, the timestamp binding module uses the synchronization pulse mark as the absolute anchor point to perform forced fine-tuning and correction on the time node of the discrete visual data, generating the corrected and aligned discrete visual data.

[0041] The timestamp binding module uses a multimedia encapsulation format to package audio data with synchronization pulse markers and corrected, aligned discrete visual data into a digital multimedia container. The audio data with synchronization pulse markers is encapsulated as the main audio track. Micro-videos and single-frame still images are encapsulated as a video track. Cursor trajectory sequences and handwritten annotation sequences are encapsulated as metadata tracks. The timestamp binding module generates the final lightweight discretized video file. This lightweight discretized video file, together with the hardware scheduling instructions generated by the hardware control module, constitutes the image device control and video stream discretization processing results based on frame change detection of this device.

[0042] The technical solutions in the embodiments of the present invention will be described in detail below:

[0043] This invention provides an image device control and video stream discretization processing apparatus based on frame change detection, which can be flexibly integrated and applied across platforms, devices, and scenarios.

[0044] In one application scenario, this device can be integrated as a core module into a shooting device with image geometry correction capabilities, performing discretization processing of audio and video streams while completing optical distortion correction.

[0045] In another application scenario, this device can be independently integrated into image acquisition terminals such as smartphones, gimbal cameras, or conventional live streaming cameras. By analyzing the raw data stream during the recording process in real time, it can achieve dynamic optimization of the device's operating power consumption.

[0046] In another application scenario, this device can be deployed as a software function module or a fixed program on computing terminals such as personal computers, interactive whiteboards, or cloud servers to automatically perform offline discretization reconstruction of existing videos pre-stored locally or in the cloud.

[0047] In this embodiment, the audio processing module is used to construct a global time reference. In real-time shooting mode, the audio processing module activates the pickup device to continuously collect sound sources as raw audio. The pickup device can be a condenser electret microphone, a microelectromechanical system (MEMS) microphone, or a microphone array composed of multiple sensor units. The pickup device converts the received sound wave signal into an analog electrical signal, which is then processed by a preamplifier circuit and an analog-to-digital converter into a digitized pulse-code modulation signal.

[0048] In the offline discretization processing mode, the audio processing module uses an audio separator to extract the original audio tracks from the pre-stored existing video as the raw audio. For the identification, demultiplexing, and stream extraction of audio tracks in the existing video, those skilled in the art can use a demultiplexer based on a standardized multimedia framework to perform the operations. The specific implementation methods are well-known in the field and will not be elaborated upon here.

[0049] Considering the importance of the continuity of audio information for semantic understanding in teaching recordings and PPT presentations, the audio processing module follows the principle of coherence and no deletion after acquiring the original audio.

[0050] During this processing, even when the image is static and the ambient volume is low, the audio processing module does not perform any signal amplitude-based filtering, silence detection cropping, or segmentation operations on the original audio. This ensures that the generated coherent and unedited audio data has absolute linear integrity on the timeline. This coherent and unedited processing principle provides a constant temporal anchoring background for the subsequent discretized visual components, avoiding timestamp breaks caused by audio slicing.

[0051] Based on this, in order to solve the cumulative clock drift caused by slight deviations in the sampling crystal oscillator frequency of different hardware devices, the audio processing module provides high-precision time synchronization pulses by calling the underlying real-time clock.

[0052] As a preferred approach, the audio processing module writes hardware-level synchronization pulse markers into the continuous, unedited audio data at a fixed interval set internally by the device. The fixed interval is determined based on the device's preset requirements for audio-visual synchronization tolerance and the typical temperature drift rate of the hardware clock. Its purpose is to ensure that, during recording sessions lasting several hours, the synchronization error between visual and audio data remains outside the range perceptible to the human ear. For example, the fixed interval can be set between 10 milliseconds and 2000 milliseconds.

[0053] In practice, hardware-level sync pulse markers can be constructed as private extended data in the header field of audio data frames, or inserted into the metadata track of the audio container stream without altering the original audio sample values. By periodically injecting this marker, the audio processing module ultimately generates audio data with the sync pulse marker.

[0054] Audio data with synchronization pulse markers establishes a series of absolute anchor points on its own time axis through hardware-level synchronization pulse markers. These anchor points can forcibly correct the timing deviation caused by fluctuations in image acquisition frequency, providing a reliable reference benchmark for determining the amplitude of image changes in subsequent image frames and for the timing binding of various discrete visual data.

[0055] In this embodiment, the image acquisition and parsing module receives initial frame rate commands, reduced acquisition frame rate commands, or restored normal frame rate commands from the hardware control module during the initialization phase or runtime loop. The image acquisition and parsing module can execute corresponding low-level driver actions for different received commands, thereby achieving switching of working states.

[0056] Upon receiving an initial frame rate command or a command to resume normal frame rate, the image acquisition and analysis module configures the frequency divider register or clock enable parameters of the image acquisition device to enter high-frequency acquisition mode, and acquires the video stream to be processed in high-frequency acquisition mode. High-frequency acquisition mode typically corresponds to the full frame rate mode of the image acquisition device, such as 30fps or 60fps.

[0057] When a command to reduce the acquisition frame rate is received, the image acquisition and analysis module enters a low-frequency, low-power state by changing the sensor sampling interval, increasing the number of blanking period lines, or directly performing frame skipping sampling at the hardware link layer, and acquires the video stream to be processed in the low-frequency, low-power state.

[0058] At the underlying hardware level, the command to reduce the acquisition frame rate can directly trigger the underlying driver to reconfigure the clock divider controller inside the image acquisition device. By changing the sampling interval of the image acquisition device, increasing the number of blanking lines, or directly performing frame skipping sampling at the hardware link layer, it can force the device into a low-frequency, low-power state.

[0059] In this state, the image acquisition and analysis module can reduce the active duty cycle of the image processing bus and analog-to-digital conversion circuit. The acquisition frequency in the low-frequency, low-power state is lower than the normal frame rate, and its setting range can be from 1fps to 5fps.

[0060] The aforementioned video stream to be processed includes continuous video streams acquired in real time using image acquisition devices or existing video streams read sequentially. For continuous video streams from the scene, those skilled in the art can use video capture technologies based on mobile industry processor interfaces or Universal Serial Bus protocols to acquire them in real time. The data transmission process is well-known in the field and will not be described in detail here.

[0061] The image acquisition and analysis module uses a video parser to decapsulate and decode the video stream to be processed, parsing it into a baseline continuous video stream and video frames arranged in time sequence. The baseline continuous video stream is stored in a circular buffer as a data mirror for subsequent lossless visual component extraction, while the video frames are sent to the processing queue as computational objects for image feature extraction.

[0062] The image acquisition and analysis module extracts the current frame and adjacent frames from the video frames, directly retaining the current frame as the unprocessed current frame. To reduce the computational load of subsequent inter-frame comparisons and eliminate the interference of color information on motion detection, the image acquisition and analysis module performs grayscale processing on the current frame and adjacent frames, generating grayscale-processed current frames and grayscale-processed adjacent frames. The grayscale processing is achieved by extracting the luminance components from the original pixels; in actual processing, a weighted average method can be used to calculate the red, green, and blue channels.

[0063] After grayscale conversion, the image acquisition and analysis module calculates the brightness difference between the current frame (after grayscale conversion) and its adjacent frames (after grayscale conversion). The brightness difference is calculated by iterating through the grayscale values ​​at all coordinate positions in the image, calculating the absolute value of the grayscale difference between corresponding pixels in two frames, and accumulating this to obtain the total brightness fluctuation across the entire image. The image acquisition and analysis module then averages this total brightness fluctuation to obtain the global average brightness difference. This global average brightness difference is set as the ambient light compensation value.

[0064] In this process, to avoid motion misjudgment caused by sudden changes in ambient light such as cloud cover and light flicker, the image acquisition and analysis module identifies the stability of the lighting environment by comparing the global average brightness difference with a preset illumination change threshold. The illumination change threshold is set based on the dark current noise level of the image acquisition device under a constant background and the typical fluctuation value of ambient light, and its value can be set between 5 and 60 gray levels.

[0065] If the global average brightness difference is greater than the illumination change threshold, it indicates that the brightness change is mainly caused by global illumination fluctuations rather than local object movement. In this case, the ambient light compensation value is confirmed to be enabled to offset background fluctuations in subsequent calculations. Conversely, if it is not greater than the illumination change threshold, it indicates that the current ambient light is stable. The device can dynamically reset the ambient light compensation value passed to subsequent modules to zero to prevent overcompensation from interfering with real motion detection.

[0066] In this embodiment, the frame change detection module is used to perform real-time quantitative analysis on the video sequence. It generates the frame change amplitude to characterize the intensity of the scene activity by performing multi-dimensional comparison of visual features at continuous time scales.

[0067] The frame change detection module receives the current frame after grayscale processing, adjacent frames after grayscale processing, and ambient light compensation values ​​from the image acquisition and analysis module. In its computational logic design, this module balances the overall consistency of pixel brightness with the geometric stability of object edges, thereby extracting realistic motion components in complex lighting environments.

[0068] To eliminate the overall bias in pixel brightness caused by ambient light fluctuations, as a preferred approach, the frame change detection module uses an illumination-compensated pixel difference formula to calculate the global pixel change after removing illumination interference. In this calculation, the ambient light compensation value is used as a brightness offset correction term for each spatial sampling point of the current frame after grayscale processing. This method effectively isolates non-motion grayscale shifts caused by light source fluctuations. The illumination-compensated pixel difference formula is as follows:

[0069] ;

[0070] In the formula: The global pixel variation after removing illumination interference; The sequence number of the current frame; The sequence number of the adjacent frame; Image width; Image height; The summation symbol; The x-coordinate variable for image pixels; The vertical coordinate variable for image pixels; This is the absolute value operator; This represents the pixel brightness value at the corresponding coordinates in the current frame after grayscale processing; This is the ambient light compensation value; This represents the pixel brightness value at corresponding coordinates for adjacent frames after grayscale processing.

[0071] Here, the pixel change is normalized by dividing by the product of the image width and the image height, ensuring the universality of the image change detection logic across devices with different resolutions.

[0072] Changes in the outline of objects in an image often reflect substantial shifts in the image content more effectively than changes in pixel grayscale. Therefore, while performing pixel comparison, the frame change detection module simultaneously performs structured change detection based on edge features.

[0073] The frame change detection module uses an edge extraction operator to extract edge feature maps of the current frame after grayscale processing and its adjacent frames after grayscale processing. The specific implementation of the edge extraction operator can adopt a first-order or second-order difference operator based on grayscale gradient distribution (such as the Sobel operator or the Laplace operator), which anchors the contour information of the object by identifying abrupt changes in pixel grayscale values. The calculation process of edge extraction is a well-known technology in this field and will not be described in detail here.

[0074] After binarization, the generated edge feature map clearly reflects the topological structure of the image through binary states. Subsequently, the module uses a binary edge contour comparison formula to calculate the change in edge structure of the edge feature map. This operation quantifies the degree of change in the object's geometry by accumulating the differences in edge topology between two consecutive frames. The binary edge contour comparison formula is as follows:

[0075] ;

[0076] In the formula: This refers to the change in the edge structure; This represents the pixel value at the corresponding coordinates of the edge feature map for the current frame. This represents the pixel value at the corresponding coordinates of the edge feature map corresponding to adjacent frames.

[0077] The global pixel variation calculated above after removing illumination interference reflects the overall brightness deviation of the image, while the edge structure variation focuses on the structural changes in the image content.

[0078] To improve the robustness of the device in different teaching environments, the frame change detection module assigns preset fusion weights to the global pixel change and edge structure change after removing lighting interference and performs linear weighting to calculate the image change amplitude of the current frame.

[0079] The preset blending weights are pre-set based on the application scenario's need to balance lighting stability and structural motion sensitivity. For example, in an open classroom scene with intense ambient light, the weight of edge structure variation should be increased to suppress noise; while in a well-lit indoor recording studio, the weight of pixel variation can be appropriately increased to capture subtle cursor movements. In a specific application, the preset blending weight of edge structure variation can be set between 0.4 and 0.8, and the preset blending weight of global pixel variation after removing lighting interference can be set between 0.2 and 0.6. The image change amplitude obtained through this weighted calculation will serve as the core criterion for subsequent hardware scheduling and discretization processing.

[0080] See appendix Figure 3 In this embodiment, the hardware control module is used to implement closed-loop control of the front-end acquisition hardware based on the dynamic evolution of the image. During the initialization phase, the hardware control module actively generates an initial frame rate command to instruct the image acquisition and analysis module to start the data acquisition logic at a preset standard sampling frequency. During device operation, the hardware control module continuously receives the image change amplitude from the frame change detection module and compares this quantified value with an internally preset threshold system at multiple levels to determine the physical evolution attributes of the current image.

[0081] To accurately distinguish between different activity intensities, the hardware control module is preset with static thresholds, local thresholds, and global thresholds that are in an increasing relationship.

[0082] The static threshold is set based on the residual thermal noise of the image acquisition device under static background and the minimum error fluctuation after algorithm normalization. It is used to remove invalid background noise from a physical level, and its value range can be set between 0.01 and 0.05.

[0083] The local threshold is set based on the characteristics of local pixel cluster changes caused by cursor movement in the teaching scenario and the typical energy distribution of edge topology fine-tuning. This value reflects the energy proportion of local non-coherent motion in the normalized matrix, and its value range can be set between 0.05 and 0.15.

[0084] The global threshold is set based on the intensity of pixel and edge abrupt changes across the entire layer when slide page transitions, significant content erasures, or presentation scene changes. Its value range can be set between 0.15 and 0.50.

[0085] The hardware control module can execute the following judgment logic based on the specific range of the screen change amplitude: if the comparison shows that the screen change amplitude is less than the static threshold, it indicates that the screen is absolutely still or there is only background interference. The hardware control module will indicate the current screen state as static and immediately generate a command to reduce the acquisition frame rate.

[0086] If the comparison shows that the change in the image is greater than or equal to the static threshold but less than the local threshold, it indicates that there is a small-scale activity such as cursor movement in the image. The hardware control module will indicate the current image state as local cursor movement and simultaneously generate a command to reduce the acquisition frame rate.

[0087] If the comparison shows that the change in the image is greater than or equal to the local threshold but less than the global threshold, it indicates that the speaker is performing a local continuous operation such as writing, drawing, or annotating. The hardware control module will indicate the current image status as a handwriting change and generate a corresponding instruction to reduce the acquisition frame rate.

[0088] Meanwhile, the hardware control module introduces state maintenance timing logic when determining state transitions. That is, when the screen change amplitude fluctuates at the edge of adjacent thresholds, the instruction switch must be executed only after several consecutive frames meet the determination conditions, so as to avoid the extra overhead caused by frequent hardware jumps.

[0089] When the above screen status indicators are static, local cursor movement, and handwriting changes, since the screen does not involve global content updates, the hardware control module forces the image acquisition and analysis module into a low-frequency, low-power state by issuing a command to reduce the acquisition frame rate.

[0090] At the underlying hardware implementation level, the instruction to reduce the acquisition frame rate can directly trigger the underlying driver to reconfigure the hardware registers inside the image acquisition device. This can be achieved by changing the pixel sensor sampling interval of the image acquisition device, increasing the number of vertical blanking rows (V-Blanking), or directly performing frame skipping sampling at the physical hardware link layer, thereby forcing the device into a low-frequency, low-power state.

[0091] This direct intervention based on the hardware link layer reduces the switching frequency of the analog-to-digital conversion circuit and the back-end processing bus, thereby reducing unnecessary power consumption, while ensuring the capture of necessary visual features.

[0092] If the comparison shows that the change in the image is greater than or equal to the global threshold, it is determined that a significant change has occurred, such as a slideshow or video transition, and the hardware control module indicates that the current image state has changed significantly. At this point, in order to fully capture the dynamic process of content switching, the hardware control module quickly generates a command to restore the normal frame rate.

[0093] The command to restore the normal frame rate applies to the image acquisition and analysis module, forcing the underlying camera hardware to immediately remove the frequency division limit and restore the acquisition to a high-frequency state.

[0094] Static pauses, partial cursor movement, handwriting changes, and significant changes together constitute the screen status indicator. This indicator is not only used to drive hardware frequency switching, but is also synchronously output to the storage control module to guide the subsequent hierarchical extraction of various discrete visual data.

[0095] In this embodiment, the storage control module is used to synchronously receive screen status indications from the hardware control module, as well as unprocessed current frames and reference continuous video streams from the image acquisition and analysis module. The storage control module is used to perform on-demand truncation, logical discarding, or feature space transformation on the massive image data stream according to the screen status indications, thereby reconstructing the continuous video signal into discrete visual data composed of discrete components of different dimensions.

[0096] In the specific extraction process, the storage control module executes corresponding extraction logic based on the screen status indication. When the screen status indication is static, it indicates that the screen content is highly redundant at the physical level. As a preferred approach, the storage control module only extracts and saves a single static frame from the received, unprocessed current frame. After performing this operation, the storage control module actively stops writing subsequent video frames to disk until the screen status indication changes. This processing logic significantly reduces the I / O load on the storage medium by compressing the redundant video frame sequence into independent bitmap files.

[0097] When the screen status indicates local cursor movement, only a very small area of ​​pixel change exists in the image. At this point, the storage control module chooses to discard the entire current frame and instead uses a feature extraction algorithm to identify the cursor position in the unprocessed current frame.

[0098] To achieve accurate vectorization and noise reduction, the storage control module first performs a pixel XOR operation on the unprocessed current frame and the reference frame from the previous moment to extract the connected components formed by the changed pixels. Then, it extracts the geometric bounding rectangle of the connected components and performs geometric rule matching between the aspect ratio and area of ​​the geometric bounding rectangle and the cursor size feature model preset by the system. If the matching error is within the set matching range, it filters out pseudo-connected components caused by screen reflection or small noise, and calculates the centroid pixel coordinates of the effective connected components, which are used as the cursor position in the absolute coordinate system.

[0099] When generating the cursor trajectory sequence, the storage control module extracts the cursor positions from multiple consecutive frames along with their corresponding timestamps, and records them using a time-series encoding method according to a preset vector data format. This time-series encoding record only retains the initial absolute coordinates, the coordinate displacement increment values ​​of subsequent frames relative to the initial coordinates, and the corresponding time offsets.

[0100] By converting image frames into temporally sequenced vector coordinates through geometric feature matching, the data space occupied is reduced to the byte level.

[0101] When the screen status indicates a handwriting change, the storage control module performs incremental extraction based on the presenter's writing actions on the courseware. This module only extracts handwriting points belonging to the newly added pixel set from the unprocessed current frame.

[0102] In practical applications, the storage control module uses the background difference method to extract the difference region between the current frame and the background reference frame, and filters environmental noise according to the preset handwriting grayscale threshold.

[0103] The handwriting grayscale threshold is set based on the contrast characteristics of the teaching background color and commonly used handwriting colors. It is used to eliminate artifacts caused by gain fluctuations in the image acquisition device. The handwriting grayscale threshold can be set between 10 and 120 gray levels. Since the image handwriting may be in a low-frequency, low-power state of 1fps to 5fps in this state, the directly extracted points may have trajectory breaks. Therefore, after extracting the basic points, the storage control module can combine spline curves or linear interpolation algorithms to smoothly complete the missing coordinates in adjacent time periods. By temporally encoding and recording these pixels that meet the threshold conditions and interpolation, the storage control module generates a visually continuous sequence of handwritten annotations.

[0104] In situations where the scene undergoes rapid changes, i.e., when the scene status indicator shows significant shifts, the storage control module, in order to fully preserve the dynamic continuity of the scene, completely extracts the baseline continuous video stream from the high-frequency acquisition state and generates a micro-video. Micro-videos typically exist in the form of short video clips, recording the complete process of events such as slideshow animations or native multimedia dynamic effects. Furthermore, if the storage control module detects that a device has accessed the presentation screen through the underlying video memory monitoring interface or video capture driver—for example, through a video signal input via a high-definition multimedia interface or a third-party media file playback event—the storage control module directly extracts the original source data and generates an interstitial media segment.

[0105] The storage control module collectively refers to the aforementioned generated single-frame static images, cursor trajectory sequences, handwritten annotation sequences, micro-videos, and inserted material clips as discrete visual data. To ensure that these discrete components can be reconstructed with high precision in subsequent time sequences, the storage control module records the start and end times of each discrete visual data item in real time, thereby generating the corresponding time nodes for the discrete visual data. These time nodes, serving as key metadata for the global time index, can establish a mapping relationship with audio data marked with synchronization pulses.

[0106] In this embodiment, the timestamp binding module receives audio data with synchronization pulse markers from the audio processing module, and discrete visual data and corresponding time nodes from the storage control module. In the field of audio and video processing, since the audio analog-to-digital conversion circuit and the image sensor are typically driven by independent crystal oscillators, long-term operation can lead to microsecond-level phase accumulation deviations in the logic clock. The timestamp binding module extracts the physical markers embedded in the audio stream, providing a unified time reference for discontinuous visual components, thereby eliminating audio-visual asynchrony caused by fluctuations in the hardware sampling frequency.

[0107] During the execution of the alignment logic, the timestamp binding module maps the time nodes of discrete visual data to the time axis of audio data marked with synchronization pulses.

[0108] Specifically, the timestamp binding module retrieves the synchronization pulse markers from the audio data containing synchronization pulse markers to obtain the absolute system time corresponding to each pulse point. If a time offset is found between the time node of the discrete visual data and the corresponding synchronization pulse marker, the timestamp binding module performs forced fine-tuning correction on the time node of the discrete visual data using the synchronization pulse marker as the absolute anchor point. The time offset is calculated by taking the difference between the logic counter value corresponding to the time node of the discrete visual data and the physical clock value carried by the corresponding sequence number of the synchronization pulse marker.

[0109] As a preferred approach, to avoid unnecessary jitter corrections that could deplete processing resources, the timestamp binding module determines whether the time offset exceeds a preset synchronization tolerance threshold. The synchronization tolerance threshold is set based on the limits of human hearing and vision's perception of audio-visual misalignment and the minimum buffer depth of the multimedia player; its purpose is to define an acceptable range for phase difference tolerance. The synchronization tolerance threshold can be set between 5 milliseconds and 50 milliseconds. If the time offset exceeds this threshold, the timestamp binding module rewrites the start and decode timestamps of the discrete visual data to align them with the absolute physical time defined by the synchronization pulse marker, thereby generating corrected and aligned discrete visual data.

[0110] After timing correction is completed, the timestamp binding module uses a multimedia container format to package the audio data with synchronization pulse markers and the corrected and aligned discrete visual data into a digital multimedia container. The multimedia container format can be a container protocol under the efficient video coding standard, or a standardized container format with multi-track extension capabilities.

[0111] As one specific implementation method, the timestamp binding module maps tracks according to data attributes:

[0112] Audio data marked with sync pulses is encapsulated into a main audio track; micro-videos and single-frame still images are encapsulated into a video track; and cursor trajectory sequences and handwritten annotation sequences are encapsulated into a metadata track. During the encapsulation of the metadata track, the timestamp binding module adds a type identifier to each type of data to distinguish between vector coordinates, pixel gain points, and text commands.

[0113] The lightweight discretized video file enables on-demand rendering at the playback end, allowing users to individually enable or disable the display of handwriting during playback. Ultimately, the lightweight discretized video file and the hardware scheduling instructions generated by the hardware control module together constitute the image device control and video stream discretization processing results based on frame change detection of this device.

[0114] To further illustrate the technical solution of the present invention, this embodiment takes the teaching courseware recording scenario in a smart classroom of a university as an example, and provides the following specific application embodiment:

[0115] In this application scenario, the front-end camera driven by the image acquisition and analysis module has a resolution of 1920×1080 (i.e., image width W=1920, image height H=1080). The video analyzer analyzes the continuous video stream at a normal frame rate of 30fps under normal conditions. The internal illumination change threshold is set to 30 gray levels, the static threshold is set to 0.02, the local threshold is set to 0.10, and the global threshold is set to 0.25. Regarding the preset fusion weights, the preset fusion weight for the global pixel change after removing illumination interference is 0.4, and the preset fusion weight for edge structure change is 0.6.

[0116] At the 15-minute mark of the lecture, the instructor used a stylus to draw geometric shapes on a static PowerPoint slide. At this point, the image acquisition and analysis module extracted the unprocessed current frame and its adjacent frames from the previous moment. Due to slight flickering in the classroom projector, the calculated global average brightness difference for the entire image was 12 gray levels. Since 12 is less than the illumination abrupt change threshold of 30, the device normalized this value (12 / 255) and set the ambient light compensation value to 0.047.

[0117] Subsequently, the data enters the frame change detection module for quantization calculation. In the calculation of the illumination compensation pixel difference formula, the movement of the speaker's hand and handwriting caused significant grayscale changes in approximately 186,600 pixels (about 9% of the total image area). The average absolute value of the normalized pixel brightness difference in these changed areas is 0.82; while the pixel brightness difference in the remaining background areas, after deducting the ambient light compensation value... The values ​​all then approach 0. Substituting the above data into the formula, we calculate the global pixel variation after removing illumination interference:

[0118] ;

[0119] Simultaneously, the frame change detection module performs calculations of the binary edge contour comparison formula. To ensure that the edge change conforms to a threshold system of [0,1], the device's underlying layer performs pixel value... A density-averaged representation was adopted, meaning that each pixel in the image that underwent a state flip (from edgeless to edged, or vice versa) was assigned a value. In this handwriting drawing action, the extension of the strokes and the movement of the hand outline caused approximately 269,500 edge pixels to undergo state flipping. Substituting these values ​​into the formula to calculate the change in edge structure:

[0120] ;

[0121] After obtaining the feature changes in the above two dimensions, the frame change detection module performs a linear weighted calculation to obtain the magnitude of the image change in the current frame:

[0122] ;

[0123] After receiving the image change value of 0.1076, the hardware control module compares it with its internal threshold. Since 0.1076 is greater than the local threshold of 0.10 and less than the global threshold of 0.25, the hardware control module accurately determines the current image status as a handwriting change. Based on this indication, the hardware control module immediately generates a command to reduce the acquisition frame rate and sends it to the front-end acquisition hardware, instantly lowering the acquisition frame rate from 30fps to a low-frequency, low-power state of 5fps. Simultaneously, the storage control module abandons full-area writing of these 5fps video frames to disk, instead using background subtraction to extract only the coordinates of newly added strokes in the image, generating a low-data-volume handwritten annotation sequence, and anchoring it in time with audio data marked with synchronization pulses through a timestamp binding module.

[0124] See appendix Figure 4 , attached Figure 4 This demonstrates the macroscopic control effect of the device in the aforementioned application scenarios, specifically the dynamic correspondence between the magnitude of image changes and the device's storage increment rate over time. (Appendix) Figure 4 The horizontal axis represents the teaching time axis (unit: seconds), and the vertical axis includes two axes: the left main vertical axis represents the range of screen changes, and the right secondary vertical axis represents the device storage increment rate (unit: KB / s).

[0125] Appendix Figure 4 The solid line represents the curve of real-time calculated image change amplitude; the dashed line with data point markers represents the corresponding device storage increment rate curve. (Appendix) Figure 4 The key decision thresholds are marked by three horizontal dashed lines, corresponding from bottom to top to a static threshold of 0.02, a local threshold of 0.10, and a global threshold of 0.25.

[0126] Combined with appendix Figure 4 As shown, in the 0 to 10 second interval, when the teacher presents the PPT and the screen remains still, the solid line fluctuates slightly around 0.01, failing to reach the static threshold of 0.02, indicating that the device is in a static state. At this time, the corresponding marked dashed line is at a low reference position, retaining only the minimal overhead of audio data and a single static frame, with a device storage increment rate of approximately 15KB / s.

[0127] Between 10 and 25 seconds, the teacher writes a graphic, and the solid line jumps to around 0.11 to 0.12 (corresponding to the handwriting change phase fluctuating around 0.1076 as calculated above). This value exceeds the local threshold of 0.10 but does not reach the global threshold of 0.25. At this time, the dashed line only shows a slight rise, and the device's storage increment rate remains at around 40KB / s. Because only the vector sequence of coordinate points is extracted and stored in this state, the huge storage overhead of conventional video recording is avoided.

[0128] Between 25 and 35 seconds, the teacher completes writing the geometric shape and pauses the action, the image returning to a stable state without significant movement. The solid line quickly drops below 0.02, again classifying it as a static state, while the corresponding dashed line also simultaneously drops back to the baseline data recording state of approximately 15KB / s.

[0129] At 35 seconds, the teacher switched to the next slide in the PPT. The solid line momentarily spiked above 0.30, exceeding the global threshold of 0.25, triggering a significant change. At that instant, the dashed line also experienced a sharp jump, soaring to 2500KB / s. This indicates that the hardware control module had forced the device into a high-frequency acquisition state, and the storage control module was fully capturing this micro-video of the slide transition. After the PPT transition was complete, as time progressed to 40 seconds, both the solid and dashed lines returned to their bottom baseline state.

[0130] The above application examples demonstrate that this solution, through a multi-level threshold algorithm, achieves orders-of-magnitude compression of storage space and device power consumption while ensuring that effective teaching information is not lost.

[0131] To further verify the actual effect of the technical solution of the present invention, the following provides detailed experimental verification and effect comparison in combination with real teaching recording scenarios.

[0132] Experimental verification steps:

[0133] Three parallel visual acquisition and recording systems were set up in a standard smart classroom. All three systems used the same model of high-definition image acquisition device (1920×1080 resolution), and the processor and underlying operating system were kept consistent.

[0134] Control group A (traditional continuous recording equipment): Full-frame video was continuously recorded using a traditional fixed frame rate (30fps).

[0135] Control group B (basic differential device): The basic global pixel differential algorithm is used to calculate the image change amplitude, without involving ambient light compensation values ​​or calculating edge features. The judgment threshold is the same as that of the experimental group.

[0136] Experimental Group C (the device of this invention): Deployed the image device control and video stream discretization processing device based on frame change detection of this invention. The static threshold is set to 0.02, the local threshold to 0.10, and the global threshold to 0.25. The preset fusion weight configuration is as follows: the global pixel change weight after removing illumination interference is 0.4, and the edge structure change weight is 0.6.

[0137] Design two sub-test scenarios:

[0138] Scenario 1 (Light Interference Test): Turn on the projector in the classroom, which has a slight flicker, and have the tester stand still in the monitored area for 20 minutes.

[0139] Scenario 2 (Slight Motion Test): With the projector off, the tester uses a fine-sized stylus to write geometric lines on the screen for 20 minutes.

[0140] The device's backend synchronously records the screen status indicators of control group B and experimental group C, generating a log. If the device fails to classify it as static in scenario one, it is counted as a false judgment of lighting interference; if the device fails to classify it as handwriting change in scenario two, it is counted as a missed judgment of minor movement.

[0141] The lecture will consist of a 45-minute professional course delivered by the main instructor. The teaching activities will include: PowerPoint presentations without dynamic content, partial cursor movement, handwriting changes, and significant changes (PowerPoint slide transitions). All three sets of equipment will be started simultaneously and stopped simultaneously after the lecture.

[0142] The average operating power consumption of the image acquisition and analysis modules in each device was recorded using a power analyzer. The total storage usage of the generated files was calculated after the experiment.

[0143] Three independent reviewers were invited to conduct a blind test to evaluate the retention rate of effective visual information (including the integrity of the blackboard writing, the accuracy of the cursor, and the coherence of the content) in the reconstructed and rendered images of control group B and experimental group C, using the continuous video of control group A as a benchmark.

[0144] Based on the above experimental steps, the statistical results of the comparative tests are shown in Table 1:

[0145] Table 1. Overall Effect Comparison Test Results

[0146] Test metrics Control group A (traditional continuous recording equipment) Control group B (basic differential equipment) Experimental Group C (The Device of the Invention) Light interference misjudgment rate Not applicable (no decision logic) 18.20% 0.50% Missed detection rate of subtle movements Not applicable (no decision logic) 14.60% 0.80% Total file storage usage 1855.4MB 620.5MB 210.2MB Average operating power consumption 4.82W 3.50W 2.18W Effective visual information retention rate 99.8% (benchmark reference) 85.40% 99.10%

[0147] The following conclusions can be drawn from the experimental data in Table 1:

[0148] In the flickering environment of Scene 1, the misclassification rate of control group B reached 18.2%. This is because ambient light caused an overall shift in the grayscale values ​​of background pixels across the entire image, which the basic difference algorithm identified as motion, resulting in calculated values ​​higher than the static threshold.

[0149] Experimental group C utilized a pixel difference formula for illumination compensation. During the calculation process, the absolute value of the current frame's pixel brightness value (after subtracting the ambient light compensation value) was compared with the brightness values ​​of adjacent frames, and the difference between the two values ​​was summed. This offset the overall brightness shift caused by light source fluctuations, making the calculated global pixel variation after eliminating illumination interference closer to the actual object motion value. Using this formula, experimental group C successfully reduced the false positive rate for illumination interference to 0.5%.

[0150] The binary edge contour comparison formula compensates for the lack of features in subtle movements and reduces the false negative rate.

[0151] In scenario two, for behaviors like handwriting changes where the area of ​​pixel change is extremely small, the false negative rate of control group B was as high as 14.6%. Since the result of simple pixel difference calculation usually cannot exceed the local threshold of 0.10, the device will misjudge it as static.

[0152] Experimental group C simultaneously utilized a binary edge contour comparison formula to calculate the sum of the absolute differences between the pixel values ​​at corresponding coordinates of the edge feature maps of the current frame and adjacent frames. Since the writing of fine lines causes numerous inversions of edge contour states, this formula can capture this structural change, yielding a numerically clear amount of edge structural change. By assigning this variable a preset fusion weight of 0.6 and performing linear weighting, the final image change amplitude can accurately exceed a local threshold, reducing the missed detection rate of subtle movements to 0.8%.

[0153] Based on the accurate quantification of image changes, the hardware control module can precisely generate commands to reduce the acquisition frame rate or restore the normal frame rate. Compared to the full-time operation of control group A, experimental group C forces the image acquisition and analysis module into a low-frequency, low-power state when the image is static or undergoing minor movements, resulting in an average power consumption reduction of approximately 54.7%. Simultaneously, by eliminating lighting misjudgments and accurately capturing minor movements, the storage control module can precisely extract discrete visual data based on image status indicators. Compared to control group A, experimental group C reduces total file storage usage by approximately 88.6% while maintaining a 99.1% effective visual information retention rate, fully validating the overall feasibility of the present invention.

Claims

1. An image device control and video stream discretization processing apparatus based on frame change detection, characterized in that, include: An audio processing module is used to acquire raw audio, treat the raw audio as continuous and unedited audio data, write hardware-level synchronization pulse markers into the continuous and unedited audio data, and generate audio data with synchronization pulse markers. An image acquisition and analysis module is used to acquire a video stream to be processed and parse the video stream to be processed into video frames, extract the current frame and adjacent frames from the video frames, and obtain the ambient light compensation value. The frame change detection module is used to calculate the image change amplitude based on the current frame, the adjacent frames and the ambient light compensation value, using the illumination compensation pixel difference formula and the binary edge contour comparison formula. The hardware control module is used to compare the magnitude of the image change with preset static thresholds, local thresholds and global thresholds, and generate image status indicators and hardware scheduling instructions. The storage control module is used to extract discrete visual data according to the screen status indication and record the time nodes of the discrete visual data. The timestamp binding module is used to fine-tune and correct the time nodes of the discrete visual data using the synchronization pulse marker in the audio data with synchronization pulse marker as the absolute anchor point, so as to generate a lightweight discrete video file.

2. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 1, characterized in that, The audio processing module is specifically used for: In real-time shooting scenarios, the sound pickup device is activated to collect sound sources from the scene as the original audio. In offline processing scenarios, an audio separator is used to extract the original audio track from the pre-stored existing video as the original audio. No filtering, silence detection trimming, or segmentation operations based on signal amplitude are performed on the original audio; the data that is retained as is is used as the coherent and unedited audio data. Based on the real-time clock, the hardware-level synchronization pulse marker is written into the continuous and unedited audio data at a predetermined fixed period. The fixed period is preset based on the device's requirements for audio-visual synchronization tolerance and the typical temperature drift rate of the hardware clock.

3. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 1, characterized in that, The image acquisition and analysis module is specifically used for: Receives initial frame rate commands, commands to reduce the acquisition frame rate, or commands to restore the normal frame rate from the hardware control module. When the initial frame rate instruction or the normal frame rate recovery instruction is received, the video stream to be processed is acquired in a high-frequency acquisition state; When the instruction to reduce the acquisition frame rate is received, the video stream to be processed is acquired in a low-frequency and low-power state by changing the sampling interval of the image acquisition device, increasing the number of blanking period lines, or performing frame skipping sampling at the hardware link layer. The video stream to be processed is parsed into a baseline continuous video stream and the video frames arranged in a time sequence using a video parser. Extract the current frame and the adjacent frames, retain the current frame as the unprocessed current frame, and perform grayscale processing on the current frame and the adjacent frames to generate the grayscale processed current frame and the grayscale processed adjacent frames; The brightness difference between the current frame after grayscale processing and the adjacent frames after grayscale processing is calculated to obtain the global average brightness difference, and the global average brightness difference is set as the ambient light compensation value; The video stream to be processed includes a continuous video stream of the scene acquired in real time using an image acquisition device or a stock video read sequentially.

4. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 3, characterized in that, The frame change detection module is specifically used for: Using the illumination compensation pixel difference formula, the sum of the absolute values ​​of the differences between the pixel brightness value of the current frame at the corresponding coordinates after normalization and deducting the ambient light compensation value, and the pixel brightness value of the adjacent frame at the corresponding coordinates after grayscale processing, is used to obtain the global pixel change degree after eliminating illumination interference. Edge feature maps of the current frame after grayscale processing and its adjacent frames after grayscale processing are extracted using an edge extraction operator; Using the binary edge contour comparison formula, the sum of the absolute values ​​of the differences between the pixel values ​​of the edge feature map corresponding to the current frame at the corresponding coordinates and the pixel values ​​of the edge feature map corresponding to the adjacent frame at the corresponding coordinates is calculated to obtain the edge structure change of the edge feature map; The global pixel change after removing illumination interference and the edge structure change are assigned preset fusion weights and linearly weighted to calculate the image change amplitude. The preset fusion weights are pre-set based on the application scenario's requirements for balancing illumination stability and structural motion sensitivity.

5. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 3, characterized in that, The hardware control module is specifically used for: An initial frame rate command is actively generated, and the magnitude of the image change is compared with the static threshold, the local threshold, and the global threshold. If the change in the image is less than the static threshold, the current image state is indicated as static and a command to reduce the capture frame rate is generated. If the change in the image is greater than or equal to the static threshold and less than the local threshold, the current image state is indicated as local cursor movement, and the instruction to reduce the acquisition frame rate is generated. If the change in the image is greater than or equal to the local threshold and less than the global threshold, the current image state is indicated as a handwriting change, and the instruction to reduce the acquisition frame rate is generated. If the change in the image is greater than or equal to the global threshold, the current image state is indicated as a significant change, and a command to restore the normal frame rate is generated. The hardware scheduling instructions include the initial frame rate instruction, the instruction to reduce the acquisition frame rate, and the instruction to restore the normal frame rate.

6. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 5, characterized in that, The static threshold is preset based on the residual thermal noise of the image acquisition device under static background and the minimum error fluctuation after algorithm normalization. The local threshold is preset based on the characteristics of local pixel cluster changes caused by cursor movement in the teaching scenario and the typical energy distribution of edge topology fine-tuning. The global threshold is preset based on the intensity of pixel and edge abrupt changes at the entire layer level when slide page switching, significant content erasure, or presentation scene switching.

7. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 5, characterized in that, The storage control module is specifically used for: Simultaneously receive the unprocessed current frame and the reference continuous video stream; When the screen state is indicated as static, only a single static frame is captured and saved from the unprocessed current frame, and subsequent video frames are discarded; When the screen state indicates that the local cursor is moving, the entire current frame is discarded, and the cursor position is extracted only from the unprocessed current frame to generate a cursor trajectory sequence. When the screen status indicates that the handwriting has changed, only the handwriting points are extracted from the unprocessed current frame to generate a handwritten annotation handwriting sequence. When the image status indicates a significant change, the reference continuous video stream in the high-frequency acquisition state is completely captured to generate a micro-video.

8. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 7, characterized in that, The storage control module is also used for: If a demonstration screen accessed by an image device is detected, then insert and generate interstitial material segments; The generated single-frame static image, the cursor trajectory sequence, the handwritten annotation sequence, the micro-video, and the inserted material fragments are used as the discrete visual data, and the time nodes of the discrete visual data are recorded.

9. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 8, characterized in that, The timestamp binding module is specifically used for: Map the time nodes of the discrete visual data onto the time axis of the audio data marked with synchronization pulses; Retrieve the synchronization pulse marker from the audio data with the synchronization pulse marker; If a time offset is found between the time node of the discrete visual data and the nearest synchronization pulse marker, the time node of the discrete visual data is fine-tuned and corrected using the synchronization pulse marker as the absolute anchor point to generate corrected and aligned discrete visual data.

10. The image device control and video stream discretization processing apparatus based on frame change detection according to claim 9, characterized in that, The timestamp binding module is also used for: The audio data with synchronization pulse markers and the corrected and aligned discrete visual data are packaged and written into a digital multimedia container using a multimedia encapsulation format. The audio data with synchronization pulse markers is encapsulated into a main audio track, the micro-video and the single-frame static image are encapsulated into a video track, and the cursor trajectory sequence and the handwritten annotation sequence are encapsulated into a metadata track to generate the lightweight discretized video file.

Images