A hybrid encoding processing method, system, and device based on a video engine

Abstract

This application relates to the field of hybrid coding in video engines, providing a hybrid coding processing method, system, and device based on a video engine. This includes intelligently switching between H.265 inter-frame predictive coding and MJPEG intra-frame compression coding modes by dynamically monitoring the proportion of moving areas in the video frame. When the scene is static, discrete cosine transform is used to compress spatial redundancy; when there is rapid motion, motion vector compensation is used to eliminate temporal redundancy. Simultaneously, pixel difference analysis and perceptual hashing algorithms are used to dual-filter keyframes, eliminating visually redundant duplicate frames. Finally, a timestamp indexing system is used to establish spatiotemporal correlation between the optimized dual-stream data, achieving efficient storage and accurate reconstruction of the hybrid bitstream. This solves the problems of low coding efficiency, lack of adaptive coding modes, unoptimized redundant frames, and insufficient bitstream management.

Description

Technical Field

[0001] This application relates to the field of hybrid coding technology for video engines, and more specifically, to a hybrid coding processing method, system, and device based on a video engine. Background Technology

[0002] The content in this section provides only background information related to this application and may not constitute prior art.

[0003] With the rapid development of video technology, high-definition and ultra-high-definition video are increasingly widely used in fields such as surveillance, media transmission, and terminal display. This places higher demands on the efficiency of video coding, the control of storage and transmission costs, and the ability to adapt to different scenarios. Traditional video coding technologies are mostly based on Discrete Cosine Transform (DCT), such as the MPEG-2 standard. It uses intra-frame DCT to compress spatial redundancy and inter-frame motion compensation to eliminate temporal redundancy, becoming the mainstream solution for early high-resolution video (such as HDTV) processing. However, this type of technology faces challenges when adapting to low-resolution devices at the decoding end. If the decoded high-resolution spatial domain image is directly filtered and sampled, a large number of multiplication and addition operations are required, resulting in high hardware complexity and low processing efficiency. To address this issue, Chinese Patent CN1203500A discloses a frequency domain filtering method for down-transforming discrete cosine transform coded images. This method targets high-resolution coded video under the MPEG-2 standard. Before the inverse discrete cosine transform (IDCT), it performs frequency domain low-pass filtering on the DCT coefficients, weights the selected DCT coefficients using a block mirror filter, and then converts them into spatial domain sampled values ​​via IDCT and samples them. Simultaneously, it converts motion vectors according to the sampling rate to adapt to motion compensation of low-resolution reference frames, effectively reducing the hardware cost of spatial domain filtering and realizing the adaptation and decoding of high-resolution video to low-resolution devices.

[0004] However, with the diversification of video scenarios and the upgrading of encoding requirements, the aforementioned traditional solutions and frequency domain filtering techniques have gradually revealed significant limitations. Firstly, there are bottlenecks in encoding standards and efficiency. The CN1203500A solution, based on the MPEG-2 standard, has low compression efficiency, making it difficult to meet the demands of high-definition video for low bitrates and high image quality. Furthermore, it only focuses on the downward resolution transformation of a single bitstream, failing to adaptively select the encoding mode based on the motion characteristics of the video content. This results in redundancy caused by inter-frame coding in static scenes, while lacking a targeted and efficient inter-frame redundancy elimination mechanism in dynamic scenes. Secondly, bitstream management capabilities are insufficient. Existing technologies only handle the transformation and decoding of a single bitstream, and cannot adapt to different scenarios during decoding, making it difficult to meet the coordinated requirements of dynamic image quality and static compression in modern video systems. Against this backdrop, there is an urgent need for a hybrid encoding processing method, system, and device based on a video engine. Summary of the Invention

[0005] To address the aforementioned technical problems, this application aims to provide a hybrid coding processing method, system, and device based on a video engine. By adaptively switching between the H.265 main bitstream and the MJPEG sub-bitstream according to the proportion of the motion area, combining the average absolute difference to mark duplicate frames and the perceptual hash to select keyframes to delete redundant frames, and then achieving efficient reconstruction of the hybrid bitstream through timestamp indexing and bidirectional correlation indexing, the problems of low coding efficiency, lack of adaptive coding mode, unoptimized redundant frames, and insufficient bitstream management are solved.

[0006] The objective of this application is achieved through the following technical solution:

[0007] In a first aspect, the present invention provides a hybrid encoding processing method based on a video engine, comprising:

[0008] The video engine is started and the H.265 main stream encoder and MJPEG sub-stream encoder are loaded simultaneously. The H.265 main stream encoder is enabled by default and configured to match the corresponding pixel block in the reference frame within a preset search range for each pixel block of the encoded frame, calculate the motion vector between the pixel blocks, and then perform compensation processing on the pixel blocks based on the motion vector to eliminate inter-frame redundancy. The MJPEG sub-stream encoder preloads the JPEG quantization table and keeps it in standby mode.

[0009] Initialize the preset image recognition algorithm, including setting a first threshold for the proportion of the moving area and a threshold for the average absolute difference of pixels;

[0010] Based on the initialized image recognition algorithm, the proportion of the moving region area is calculated by edge detection and optical flow. If the proportion of the moving region area is lower than the first threshold, the MJPEG sub-stream encoder is triggered, and discrete cosine transform is used to compress the spatial redundancy data of a single frame. The timestamp index is recorded synchronously when encoding is switched. If the proportion of the moving region area exceeds the first threshold, an H.265 encoding instruction is sent, and inter-frame prediction is enabled to calculate the residual between the current pixel unit and the prediction unit in the current frame.

[0011] Calculate the mean absolute difference of corresponding pixels in adjacent static frames. If the mean absolute difference is lower than the mean absolute difference threshold and the residuals in the same timestamp interval are all lower than the mean absolute difference threshold, then it is marked as a duplicate frame. Generate the perceptual hash value for each frame and compare the Hamming distance of the perceptual hash values ​​to confirm the key frame. If the key frame satisfies that the Hamming distance with the preceding and following frames exceeds the second threshold, retain the non-duplicate frames whose hash difference exceeds the second threshold and discard the rest.

[0012] The H.265 encoded video stream is filtered, and the H.265 encoded data corresponding to non-repeating frames is written into the video database according to the GOP structure. At the same time, key frames and their corresponding perceptual hash values ​​are stored in the image storage pool. During storage, a bidirectional correlation index is established by combining the timestamp index and the timestamp corresponding to each non-repeating frame. During decoding, a hybrid bitstream containing the H.265 main bitstream and the MJPEG sub-bitstream is reconstructed based on the bidirectional correlation index.

[0013] Furthermore, the preset search range of the H.265 main stream encoder is configured to be dynamically adjustable, including: identifying texture-rich regions and smooth regions in the image; wherein, texture-rich regions are regions within a preset-sized image block where the brightness value change frequency of adjacent pixels exceeds a majority proportion and the change amplitude exceeds a third threshold; smooth regions are regions within a preset-sized image block where the brightness value change frequency of adjacent pixels is lower than a preset proportion and the change amplitude is lower than a fourth threshold; the search range for texture-rich regions is expanded to a first preset range; and the search range for smooth regions is narrowed to a smaller coverage area of ​​a second preset range.

[0014] Furthermore, the step of synchronously recording the timestamp index during encoding switching specifically includes:

[0015] When switching encoding methods, the timestamp information of the main stream and the sub-stream is recorded simultaneously to form a dual timestamp mapping table; the timestamps are redundantly backed up using a dual storage mechanism of local cache and engine memory; in the subsequent decoding stage, the temporal continuity at the point of stream switching is ensured by comparing the dual timestamp mapping table.

[0016] Furthermore, the step of enabling inter-frame prediction to calculate the residual between the current pixel unit and the prediction unit in the current frame specifically includes:

[0017] Determine the reference frame set for the current coded frame, which includes at least one frame before the current coded frame, at least one frame after the current coded frame, or a combination thereof; divide the current coded frame into multiple pixel units, and for each pixel unit, perform a matching search in the corresponding reference frame to obtain a reference pixel unit that matches the pixel unit; generate a prediction unit for the current pixel unit based on the reference pixel units; calculate the residual between the current pixel unit and the prediction unit.

[0018] Furthermore, after enabling inter-frame prediction to calculate the residual between the current pixel unit and the prediction unit in the current frame, it also includes:

[0019] When performing a matching search for each pixel unit in the corresponding reference frame, the initial motion vector is determined by the positional difference between the pixel unit and the matched reference pixel unit;

[0020] The initial motion vector is smoothed. Based on the overall trend of the initial motion vector of the current pixel block and the initial motion vector of the adjacent pixel blocks, abnormal motion vectors that deviate from the overall trend by a preset angle are removed, and the abnormal motion vectors are replaced by the average trend of the initial motion vectors of the adjacent pixel blocks.

[0021] If the motion vector of the current pixel block is not continuous with the motion vectors of its neighboring pixel blocks, then the motion vector of the current pixel block is corrected according to the changing trend of the motion vectors of its neighboring pixel blocks, so that the motion vectors of its neighboring pixel blocks change continuously.

[0022] Furthermore, the formula for calculating the average absolute difference of pixels at corresponding positions in adjacent still image frames is as follows:

[0023]

[0024] Where MAD is the mean absolute difference; M is the height of the pixel block; N is the width of the pixel block; i is the row index within the pixel block; j is the column index within the pixel block; I a (i,j) represents the brightness or grayscale value of the pixel located in the i-th row and j-th column of the previous frame; b (i,j) represents the brightness or grayscale value of the pixel located in the i-th row and j-th column in the next frame.

[0025] Furthermore, the keyframes are stored in the image storage pool, specifically including:

[0026] Keyframes are divided into core keyframes and auxiliary keyframes according to their importance. Core keyframes are video frames at scene transitions, while auxiliary keyframes are action frames within the scene. Core keyframes are stored using lossless compression, while auxiliary keyframes are stored using lossy compression. A keyframe index tree is built with time order as the vertical axis and scene category as the horizontal axis to achieve fast retrieval.

[0027] Secondly, the present invention provides a hybrid encoding processing system based on a video engine, comprising:

[0028] The video engine loading module is used to start the video engine and synchronously load the H.265 main stream encoder and the MJPEG sub-stream encoder. This includes: the H.265 main stream encoder is enabled by default and configured to match the corresponding pixel block in the reference frame within a preset search range for each pixel block of the encoded frame, calculate the motion vector between pixel blocks, and then perform compensation processing on the pixel blocks based on the motion vector to eliminate inter-frame redundancy; the MJPEG sub-stream encoder preloads the JPEG quantization table and keeps it in standby mode.

[0029] An initialization module is used to initialize a preset image recognition algorithm, including setting a first threshold for the proportion of the moving region area and a threshold for the average absolute difference of pixels.

[0030] The motion region area ratio determination module is used to calculate the motion region area ratio based on the initialized image recognition algorithm through edge detection and optical flow. If the motion region area ratio is lower than the first threshold, the MJPEG sub-stream encoder is triggered to compress the spatial redundancy data of a single frame using discrete cosine transform, and the timestamp index is recorded synchronously during encoding switching. If the motion region area ratio exceeds the first threshold, an H.265 encoding instruction is sent to enable inter-frame prediction to calculate the residual between the current pixel unit and the prediction unit in the current frame.

[0031] The non-repeating frame extraction module is used to calculate the average absolute difference of corresponding pixels in adjacent static frames. If the average absolute difference is lower than the average absolute difference threshold and the residuals in the same timestamp interval are all less than the average absolute difference threshold, it is marked as a repeating frame. The perceptual hash value of each frame is generated, and the Hamming distance of the perceptual hash values ​​is compared to confirm the key frame. If the key frame satisfies that the Hamming distance with the preceding and following frames exceeds the second threshold, the non-repeating frame with a hash difference exceeding the second threshold is retained, and the remaining frames are discarded.

[0032] The hybrid bitstream generation module is used to filter the H.265 encoded video stream, write the H.265 encoded data corresponding to non-repeating frames into the video database according to the GOP structure, and store keyframes and their corresponding perceptual hash values ​​into the image storage pool. During storage, a bidirectional correlation index is established by combining the timestamp index and the timestamps corresponding to each non-repeating frame. During decoding, a hybrid bitstream containing the H.265 main bitstream and the MJPEG sub-bitstream is reconstructed based on the bidirectional correlation index.

[0033] Thirdly, the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps corresponding to the method in the first aspect.

[0034] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps corresponding to the method in the first aspect.

[0035] In summary, the technical solutions of this application have at least the following advantages and beneficial effects:

[0036] This invention intelligently switches between H.265 main stream and MJPEG sub-stream encoding modes by dynamically detecting the area ratio of moving regions in a video frame: when the scene is static, MJPEG compression is used to eliminate spatial redundancy in single frames; when there is rapid motion, H.265 inter-frame prediction is activated to eliminate temporal redundancy. The system accurately marks duplicate frames by calculating the mean absolute difference of pixels and filters key frames using the Hamming distance of perceptual hash values, achieving automated filtering and removal of redundant frames. Finally, through timestamp indexing and bidirectional associative indexing technology, optimized non-duplicate frames are stored in a GOP structure, with key frames archived separately, enabling efficient reconstruction of the mixed stream during decoding. This adaptive encoding strategy significantly improves compression efficiency and solves the problems of low efficiency, insufficient handling of redundant data, and chaotic management of mixed streams associated with traditional fixed encoding modes. Attached Figure Description

[0037] Figure 1 A flowchart of a hybrid encoding processing method based on a video engine provided by the present invention;

[0038] Figure 2 This is a logic block diagram of a hybrid encoding processing method based on a video engine in this invention;

[0039] Figure 3 A schematic diagram of the structure of a hybrid encoding processing system based on a video engine provided by the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0042] like Figure 1 and Figure 2 As shown in the embodiments of this application, a hybrid encoding processing method based on a video engine is proposed, comprising:

[0043] S101, Start the video engine and synchronously load the H.265 main stream encoder and MJPEG sub-stream encoder, including: The H.265 main stream encoder is enabled by default and configured to match the corresponding pixel block in the reference frame within a preset search range for each pixel block of the encoded frame, calculate the motion vector between pixel blocks, and then perform compensation processing on the pixel blocks based on the motion vector to eliminate inter-frame redundancy; The MJPEG sub-stream encoder preloads the JPEG quantization table and keeps it in standby state.

[0044] Specifically, the video engine is started to simultaneously load the H.265 (H.265 / High Efficiency Video Coding) main stream encoder and the MJPEG (Motion JPEG) sub-stream encoder. The "video engine" here is the control core of the entire hybrid encoding process, integrating resource scheduling, encoding instruction distribution, and data flow management functions. Its "synchronous loading" technology works by allocating independent and mutually exclusive computing resources (including dedicated memory buffers, processing units, and data interfaces) to the H.265 main stream encoder and the MJPEG sub-stream encoder through the engine's internal parallel resource allocation module. This avoids a single encoder monopolizing resources during loading, which could cause a delay in the subsequent startup of the other encoder.

[0045] During loading, the H.265 main stream encoder is enabled by default. Its core configuration focuses on the efficient elimination of inter-frame redundancy. Specifically, the encoder first divides the frame to be encoded into multiple independent pixel blocks according to a preset size (usually 16×16 pixels or 32×32 pixels. This size is chosen based on the principle of "balancing computational complexity and compression accuracy": a 16×16 pixel block can decompose the huge pixel data of the whole frame into units that can be processed efficiently, and can accurately represent local motion features, which meets the inter-frame prediction standard of H.265 encoding). Then, for each pixel block, the encoder will perform matching operations with the pixel blocks in the reference frame (usually 1-3 encoded frames before the current frame; selecting a forward reference frame can reduce data buffer pressure) within a preset search range. The initial setting of the "preset search range" here aims to balance the accuracy and computation speed of motion vector estimation. For example, it can be initially set to ±16 pixels (a rectangular area of ​​16 pixels horizontally and vertically). If the search range is too large, the amount of matching computation will increase quadratically, reducing the encoding speed; if the range is too small, the actual position of the pixel block in the reference frame may be missed, leading to an increase in matching error. Through pixel block matching, the encoder can calculate the positional offset between the current pixel block and the best matching pixel block in the reference frame. This offset is the "motion vector". After obtaining the motion vector, the encoder performs motion compensation processing based on the motion vector: using the pixel data of the matching pixel block in the reference frame, combined with the positional information indicated by the motion vector, the pixel data of the current pixel block is reconstructed. Through this process, pixel information that is repeated in the current frame and the reference frame does not need to be encoded again, thereby effectively eliminating inter-frame redundancy.

[0046] Meanwhile, the MJPEG sub-stream encoder adopts a "preloaded JPEG quantization table + standby" configuration mode. The "JPEG quantization table" is the core parameter table for MJPEG encoding to achieve single-frame spatial redundancy compression. It defines the quantization rules for frequency coefficients after performing Discrete Cosine Transform (DCT) on a single video frame: a smaller quantization step size is used for low-frequency coefficients representing image contours (preserving more details), while a larger quantization step size is used for high-frequency coefficients representing image noise or subtle textures (compressing redundant data). For example, the luminance quantization table defined in the ITU-T T.81 standard can compress single-frame data to 1 / 10-1 / 20 of the original size while ensuring subjective video quality. "Standby state" means the encoder has completed initialization and is ready to start operation immediately upon receiving encoding instructions, without requiring additional hardware initialization.

[0047] To further improve the performance of the H.265 main stream encoder, its preset search range is configured to be dynamically adjustable, including: identifying texture-rich regions and smooth regions in the image; wherein, texture-rich regions are regions within a preset-sized image block where the brightness value changes of adjacent pixels more than a majority proportion and the change amplitude exceeds a third threshold; smooth regions are regions within a preset-sized image block where the brightness value changes of adjacent pixels less than a preset proportion and the change amplitude is less than a fourth threshold; for texture-rich regions, the search range is expanded to a first preset range; for smooth regions, the search range is narrowed to a smaller coverage area of ​​a second preset range.

[0048] Specifically, the encoder first identifies texture-rich and smooth regions in the frame to be encoded. The criteria for determining texture-rich regions are: within a preset-sized image block (consistent with the pixel block size of the encoded frame, such as 16×16 pixels, to ensure compatibility between texture recognition and encoding processing), the brightness value change frequency of adjacent pixels exceeds the majority proportion (set to 60% based on a large number of video samples, meaning more than 60% of adjacent pixel pairs exhibit brightness changes), and the change magnitude exceeds the third threshold (brightness value range 0-255, the third threshold is set to 15 to balance sensitivity and noise resistance—if the threshold is too small, noise is easily misjudged as texture; if it is too large, subtle textures are easily missed). The criteria for determining smooth regions are: within the same image block, the brightness change frequency of adjacent pixels is less than 30% (below the majority proportion, forming a clear distinction), and the change magnitude is less than the fourth threshold (set to 8, accurately identifying regions with gentle brightness). The reason for choosing brightness value as the criterion is that brightness signal is the main carrier of visual information, and its changes can better reflect the density of texture. Moreover, the calculation complexity based on brightness value is lower than the joint calculation of "brightness + chromaticity", which can improve the recognition speed and ensure the real-time performance of encoding.

[0049] After completing region recognition, the encoder configures different search ranges for two types of regions: For textured regions, the search range is expanded to a first preset range (e.g., ±32 pixels). This range can cover larger motion offsets of pixel blocks (e.g., pixel block offsets when a vehicle is moving at high speed), ensuring the accuracy of motion vector estimation and reducing residual data. For example, if a pixel block of a moving vehicle is offset by 25 pixels in the reference frame, it cannot be matched if a fixed ±16 pixel range is used, but it can be matched accurately with a ±32 pixel range, reducing the amount of encoded data for that pixel block by 40%. For smooth regions, the search range is narrowed to a second preset range (e.g., ±8 pixels). Since pixel blocks in smooth regions are mostly stationary or undergoing slight motion, narrowing the range can reduce invalid calculations. For example, the calculation amount for matching pixel blocks on a stationary wall is reduced by 60% compared to a ±16 pixel range, reducing CPU utilization (from 50% to 20%) and improving encoding speed.

[0050] S102, initialize the preset image recognition algorithm, including setting a first threshold for the proportion of the moving area and a threshold for the average absolute difference of pixels.

[0051] Specifically, the "preset image recognition algorithm" employs edge detection and optical flow. Edge detection extracts the contour features of objects in the image, providing a basis for locating the boundaries of moving regions. Optical flow calculates the movement trajectory of pixels between consecutive frames, thereby quantifying the range and intensity of the moving region. The initialization process includes loading the feature extraction operators (such as the Sobel and Canny operators) required by the algorithm, setting the pyramid hierarchy parameters for optical flow calculation, and allocating memory buffers. Its core principle is to avoid encoding delays caused by the computational resources occupied during algorithm initialization during real-time encoding by pre-completing the basic configuration for algorithm operation, thus ensuring the real-time performance and accuracy of moving region detection.

[0052] The "motion area ratio" refers to the ratio of the area comprised of all moving pixels in a video frame to the total area of ​​the video frame. This ratio is a key indicator for judging the degree of motion in the image. The "first threshold" is a critical value used to define the degree of motion in the image. Its setting must be based on the specific application scenario (such as security monitoring, video conferencing, sports events, etc.) and follow the principle of "balancing motion sensitivity and encoding efficiency." For example, in security monitoring scenarios, since the image is mostly static or has slight motion, the first threshold can be set to a lower value (such as 10%). That is, when the motion area ratio is less than 10%, it is judged as a low-motion scene. In sports event scenarios, because the image motion is intense, the first threshold can be set to a higher value (such as 30%). The principle behind setting this threshold is to provide a clear basis for subsequent encoding method switching by quantifying the proportion of the motion area, avoiding erroneous encoding method switching due to ambiguity in motion degree judgment.

[0053] The "mean absolute difference of pixels" is a quantitative indicator used to measure the degree of difference between corresponding pixels in adjacent frames, while the "mean absolute difference threshold" is the critical value for determining whether adjacent frames are duplicate frames. This threshold is set based on the human eye's perception of image differences and is typically selected within the brightness value range (0-255), for example, a value between 8 and 15. When the mean absolute difference between adjacent frames is below this threshold, it indicates that the pixel differences between the two frames are extremely small and difficult for the human eye to distinguish, thus classifying them as potential duplicate frames. When it is above this threshold, it indicates that the two frames have significant differences and are not duplicate frames. By defining a critical value for pixel difference, an objective standard is provided for subsequent screening of duplicate frames, avoiding the repeated encoding and storage of frames with excessively high similarity, thereby reducing redundant data.

[0054] S103: Based on the initialized image recognition algorithm, the motion region area ratio is calculated by edge detection and optical flow method; if the motion region area ratio is lower than the first threshold, the MJPEG sub-stream encoder is triggered, and the discrete cosine transform is used to compress the spatial redundancy data of the single frame. The timestamp index is recorded synchronously when the encoding is switched; if the motion region area ratio exceeds the first threshold, the H.265 encoding instruction is sent, and inter-frame prediction is enabled to calculate the residual between the current pixel unit and the prediction unit in the current frame.

[0055] Specifically, edge detection extracts the contour features of objects in the image (such as human figures and object boundaries) by calculating the gray-level gradient of image pixels, thus defining the boundary range of moving areas and avoiding misjudging subtle gray-level changes in static backgrounds as moving areas. Optical flow tracks the positional changes of the same pixels between consecutive frames, quantifying the motion trajectory and displacement amplitude of each pixel, thereby delineating the set of all pixels with motion characteristics. By determining the spatial boundary of the moving area through edge detection and quantifying the coverage of moving pixels through optical flow, the two methods are used together to calculate the area ratio of the moving area (i.e., the ratio of the total area of ​​the set of moving pixels to the total area of ​​a single frame). Compared with single detection methods, this calculation method avoids the deficiency of edge detection in quantifying the range of motion and compensates for the problem of blurred boundary positioning in complex backgrounds by optical flow, ensuring the accuracy of the calculation of the area ratio of the moving area and providing an objective and reliable basis for subsequent switching of encoding methods.

[0056] If the calculated percentage of the motion region is lower than the first threshold, the current scene is determined to be a low-motion scene, and the MJPEG sub-stream encoder is triggered to start encoding. The principle is that in low-motion scenes, the pixel information difference between consecutive frames is extremely small, making it difficult for the H.265 main stream encoder to leverage its inter-frame prediction advantage. Instead, redundant calculations occur due to motion vector search and other operations. The MJPEG sub-stream encoder, however, is better suited to this scene due to its compression characteristics targeting single-frame spatial redundancy. It uses Discrete Cosine Transform (DCT) to convert the pixel data of a single-frame image from the spatial domain to the frequency domain, concentrating image energy on low-frequency coefficients that characterize image contours and uniform brightness. Simultaneously, combined with a pre-loaded JPEG quantization table, it uses a smaller quantization step size for low-frequency coefficients and a larger quantization step size for high-frequency coefficients that characterize image noise and subtle textures, thereby efficiently eliminating spatial redundancy in a single-frame image.

[0057] During this encoding method switching process, timestamp indices need to be recorded synchronously. Specifically, when switching from the H.265 main stream to the MJPEG sub-stream, the last valid timestamp of the H.265 main stream and the first valid timestamp of the MJPEG sub-stream are simultaneously collected to construct a dual timestamp mapping table, ensuring precise alignment of the two streams on the timeline. To prevent timestamp data loss due to hardware failures, power outages, or other unexpected events, a dual storage mechanism is employed: a local cache (e.g., an SSD) and video engine memory (e.g., DDR). The local cache is non-volatile to ensure data persistence, while the engine memory offers high-speed read / write capabilities to meet the needs of rapid access during subsequent decoding. In the subsequent decoding stage, the decoder reads this dual timestamp mapping table and compares the timestamp information of the two streams to ensure there are no time gaps or overlaps at the stream switching point, thus guaranteeing the temporal continuity of video playback.

[0058] If the area of ​​motion exceeds a first threshold, the current scene is determined to be a high-motion scene (such as athletes moving quickly in a sports event or vehicles traveling at high speed in traffic monitoring). At this point, an H.265 encoding command is sent to activate the inter-frame prediction function of the H.265 main stream encoder, calculating the residual between the current pixel unit and the prediction unit in the current frame. The principle is that in high-motion scenes, there is a large amount of repetitive pixel information between consecutive frames (such as the main pixels of moving objects and unchanging areas in the background). H.265 inter-frame prediction can generate prediction units using the pixel information of a reference frame, encoding only the difference (residual) between the current pixel unit and the prediction unit, significantly reducing the amount of data. Specifically, the residual calculation process for inter-frame prediction follows this logic: First, determine the reference frame set for the current coded frame. The reference frame set includes at least one frame before the current coded frame, at least one frame after the current coded frame, or a combination thereof. For example, for low-to-medium speed motion scenes (such as pedestrians walking), the reference frame set may only include one frame before the current coded frame (forward reference frame) to reduce data buffer pressure; for high-speed motion scenes (such as racing cars), the reference frame set may include one frame before the current coded frame and one frame after the current coded frame (bidirectional reference frame), improving the matching accuracy of moving pixel units through bidirectional prediction; then, divide the current coded frame into segments according to a preset size. Multiple independent pixel units are used. For each pixel unit, a matching search is performed in the corresponding reference frame based on the set dynamic search range to find the reference pixel unit whose pixel distribution is most similar to that of the current pixel unit. Based on the pixel data of the reference pixel unit, a prediction unit for the current pixel unit is generated through interpolation, copying, etc. (simulating the ideal pixel state of the current pixel unit when there is no motion deviation). Finally, the difference between the brightness value (or gray value) of the pixel at the corresponding position of the current pixel unit and the prediction unit is calculated. This difference is the residual. In the subsequent encoding process, only the residual data needs to be entropy encoded, and there is no need to encode the complete current pixel unit data.

[0059] After enabling inter-frame prediction to calculate residuals, the initial motion vector also needs to be smoothed and made continuous. The principle is that when matching and searching for each pixel unit in the corresponding reference frame, the initial motion vector (determined by the positional difference between the current pixel unit and the reference pixel unit, such as horizontal offset x and vertical offset y) may have abnormal values ​​due to factors such as image noise and background interference (such as the motion vector of a single pixel unit deviating too much from that of the surrounding pixel units). If it is directly used to generate prediction units, it will lead to prediction distortion, thereby increasing the amount of residual data and causing screen jitter. The specific processing steps include: First, for the initial motion vector of each pixel unit, a 3×3 neighborhood motion vector set is constructed. By calculating the mean and variance of the directional angles of this set, the overall trend of the motion vectors is determined (e.g., most vectors are 30° to the right in the horizontal direction). A preset angle threshold (e.g., ±15°) is set, and initial motion vectors that deviate from the overall trend by more than this threshold are identified as anomalous vectors. The mean of the neighborhood motion vectors is used to replace these anomalous vectors, thus achieving smoothing of the motion vectors. Second, the continuity of motion vector changes in adjacent pixel units (e.g., adjacent pixel units in the horizontal and vertical directions) is checked. If the amplitude difference between adjacent vectors exceeds the corresponding threshold, the abrupt motion vectors are corrected by linearly fitting the change curves of adjacent vectors, making the motion vector field present a continuous trend. Through smoothing and continuity processing, outliers and abrupt changes in the initial motion vectors are eliminated, avoiding image block effects and jitter caused by prediction unit distortion, while also reducing the amount of residual data.

[0060] S104, calculate the average absolute difference of corresponding pixels in adjacent static frames. If the average absolute difference is lower than the average absolute difference threshold and the residuals in the same timestamp interval are all less than the average absolute difference threshold, then mark it as a duplicate frame. Generate the perceptual hash value of each frame and compare the Hamming distance of the perceptual hash values ​​to confirm the key frame. If the key frame satisfies that the Hamming distance with the preceding and following frames exceeds the second threshold, retain the non-duplicate frames whose hash difference exceeds the second threshold, and discard the rest of the frames.

[0061] Specifically, "adjacent static frames" refers to consecutive frames identified by S103 as low-motion scenes (the area of ​​motion is less than the first threshold). In such scenes, the overall changes in the image are slight, easily resulting in a large number of frames with repetitive content. "The average absolute difference of corresponding pixels" refers to calculating the absolute difference of the brightness (or grayscale value) of pixels at the same coordinate position in two frames, and then taking the arithmetic mean of the absolute differences at all positions. This indicator can objectively reflect the overall similarity of two frames at the pixel level—the smaller the average absolute difference, the closer the content of the two frames is.

[0062] If the mean absolute difference (MAD) is lower than the MAD threshold, and the residuals within the same timestamp interval are all lower than the MAD threshold, then the frame is marked as a duplicate. This judgment logic employs a dual verification mechanism to avoid misjudgments caused by a single indicator: First, an MAD lower than the threshold indicates that the two frames are visually very similar and difficult for the human eye to distinguish; second, "residuals within the same timestamp interval" refers to the residual data generated during H.265 encoding within the same time range as the current frame (i.e., the difference between the current pixel unit and the prediction unit). If these residuals are all lower than the MAD threshold, it further confirms the high degree of repetition of the two frames from the perspective of encoded data. Through the constraint of these dual conditions, the accuracy of duplicate frame identification can be significantly improved, avoiding the misjudgment of false difference frames caused by noise or minor fluctuations in light as valid frames, thereby more accurately filtering out redundant data.

[0063] Generating a perceptual hash value for each frame includes: converting the frame image to grayscale and reducing it to a preset size (e.g., 8×8 pixels) to remove detail interference; calculating the discrete cosine transform (DCT) of the grayscale values ​​to extract low-frequency components (preserving the overall image features); taking the mean of the DCT coefficients as a threshold to binarize the coefficients; and finally obtaining a 64-bit perceptual hash value, thus achieving efficient digital representation of image content.

[0064] Keyframes are identified by comparing the Hamming distance between perceptual hash values. The Hamming distance refers to the number of different bits in the binary representation of two perceptual hash values; its magnitude directly reflects the degree of difference in the core content of the two frames—a larger Hamming distance indicates a more significant change in content. Keyframes, as markers of content change in a video sequence, must have a Hamming distance exceeding a second threshold with preceding and following frames. This second threshold is based on the minimum significant change in video content (e.g., set to 10, meaning a significant change is considered to have occurred when the Hamming distance exceeds 10). The principle is to quantify hash differences to ensure that keyframes accurately mark important turning points in the video content, providing crucial indexes for subsequent decoding and retrieval. Non-repeating frames with hash differences exceeding the second threshold are retained, while other frames are discarded, thus maximizing the removal of redundant data.

[0065] It should be noted that "remaining frames" include duplicate frames and frames with minor changes whose hash differences do not exceed the second threshold. Their content can be recovered by interpolation or reuse of adjacent retained frames, and discarding them will not affect the subjective quality of the video.

[0066] In addition, the formula for calculating the average absolute difference of pixels at corresponding positions in adjacent still image frames is:

[0067]

[0068] Where MAD is the mean absolute difference; M is the height of the pixel block; N is the width of the pixel block; i is the row index within the pixel block; j is the column index within the pixel block; I a (i,j) represents the brightness or grayscale value of the pixel located in the i-th row and j-th column of the previous frame; b (i,j) represents the brightness or grayscale value of the pixel located in the i-th row and j-th column in the next frame.

[0069] S105: Filter the H.265 encoded video stream, write the H.265 encoded data corresponding to non-repeating frames into the video database according to the GOP structure, and store key frames and their corresponding perceptual hash values ​​into the image storage pool; during storage, establish a bidirectional association index by combining the timestamp index and the timestamps corresponding to each non-repeating frame; during decoding, reconstruct the mixed bitstream containing the H.265 main bitstream and the MJPEG sub-bitstream based on the bidirectional association index.

[0070] Specifically, the H.265 encoded data corresponding to the filtered non-repeating frames is written into the video database according to the GOP structure. GOP (Group of Pictures) is a coding unit defined in the H.265 coding standard. The principle of storing according to the GOP structure is as follows: the core advantage of H.265 encoding is to reduce the amount of data through inter-frame dependencies, and the GOP structure is the basis for carrying this dependency. Storing according to this structure can ensure the structured organization of encoded data, so that the subsequent decoding process can correctly reconstruct the video frame sequence based on the inter-frame association logic within the GOP. Its beneficial effect is to ensure the storage standardization of H.265 encoded data and facilitate the decoding end to quickly locate the inter-frame reference relationship.

[0071] Simultaneously, keyframes and their corresponding perceptual hash values ​​are stored in the image storage pool. A "keyframe" is a frame identified in step S104 whose Hamming distance from the perceptual hash values ​​of preceding and following frames exceeds a second threshold. It carries core change information about the video content (such as scene transitions or frames where important actions occur). The "perceptual hash value" is a 64-bit binary data generated in step S104, providing a unique digital representation of the keyframe content. The principle behind this storage operation is that keyframes are the "semantic anchors" of the video content. Storing them separately in the image storage pool allows for rapid access without the entire video stream. The perceptual hash value, as the "digital ID card" of the keyframe, can be directly used for subsequent secondary applications such as keyframe comparison, repeated scene detection, and content retrieval. Its beneficial effect is breaking the limitation of traditional video retrieval requiring traversal of the entire bitstream, significantly improving the access efficiency of keyframes, and providing a data foundation for extended functions such as video content analysis and intelligent retrieval.

[0072] During storage, a bidirectional association index is established by combining the timestamp index and the timestamps corresponding to each non-repeating frame. The "timestamp index" is a dual timestamp mapping table (containing the timestamp correspondence between the H.265 main stream and the MJPEG sub-stream) synchronously recorded during the encoding mode switching in step S103. The "timestamps corresponding to each non-repeating frame" are the precise time identifiers generated for each non-repeating frame during the encoding stage. The "bidirectional association index" is used to construct a bidirectional mapping relationship between "timestamp and storage location". On the one hand, the storage address of the corresponding non-repeating frame in the video database can be found by the target timestamp. On the other hand, the storage address of the non-repeating frame can be traced back to its corresponding timestamp information. This solves the limitation of traditional unidirectional indexes that can only find frames by time or only find time by frames. It ensures that the required frame data can be quickly located by time dimension during decoding, or its time context can be traced by frame data, improving the flexibility and response speed of data retrieval.

[0073] During decoding, a hybrid bitstream containing the H.265 main bitstream and the MJPEG sub-bitstream is reconstructed based on a bidirectional association index. The reconstruction principle is as follows: The decoder first reads the bidirectional association index and determines the corresponding timestamp interval based on the target playback time period; it then queries the bidirectional association index through this timestamp interval to locate the H.265 bitstream data (non-repeating frames stored in the video database) and the MJPEG bitstream data (encoded data generated in the low-motion scene in step S103, whose timestamps have been included in the dual timestamp mapping table) within that interval; subsequently, the two types of bitstream data are processed in frame-level order according to the order of their timestamps. The H.265 bitstream and MJPEG substream are arranged in a continuous hybrid stream, with the H.265 bitstream corresponding to high-motion scene frames (where the area of ​​motion exceeds the first threshold) and the MJPEG bitstream corresponding to low-motion scene frames (where the area of ​​motion is below the first threshold). The two types of bitstreams are precisely aligned using timestamps. The beneficial effect of this reconstruction process is to achieve seamless fusion of the H.265 main bitstream and the MJPEG substream, avoiding playback stuttering, time gaps, or image overlap caused by bitstream switching, ensuring the smoothness and temporal continuity of video playback, while fully leveraging the compression advantages of the two encoders in different scenarios, balancing decoding quality and efficiency.

[0074] Furthermore, the specific implementation of storing keyframes in the image storage pool includes the following three parts: First, keyframes are divided into core keyframes and auxiliary keyframes according to their importance. "Core keyframes" are defined as video frames at scene transitions (such as frames where the monitoring screen switches from "empty classroom" to "students entering the classroom"). These frames carry structural changes in video content and are the core basis for video semantic segmentation, thus having the highest importance. "Auxiliary keyframes" are defined as action frames within the scene (such as frames where "students are walking around in the classroom"). These frames, although reflecting content changes, do not alter the scene structure and have a lower importance. The principle of this division is based on the differences in the semantic importance of video content, achieving hierarchical management of keyframes and avoiding resource waste or insufficient protection of important data caused by using a unified storage strategy for all keyframes. Second, lossless compression (such as PNG compression algorithm) is used for core keyframes to ensure the accuracy of subsequent operations such as scene analysis, image comparison, and intelligent recognition based on core keyframes. Lossy compression (such as JPEG compression algorithm based on the JPEG quantization table preloaded in step S101) is used for auxiliary keyframes. While ensuring acceptable image quality to the human eye, this reduces data volume by quantizing high-frequency coefficients, balancing image quality and storage overhead. This achieves a reduction in the storage capacity of auxiliary keyframes while maintaining the quality of core data, thus improving the overall storage efficiency of the image storage pool. Third, a keyframe index tree is established, with time order as the vertical axis and scene category as the horizontal axis, enabling fast retrieval. The "time order vertical axis" refers to arranging all keyframes in chronological order according to their encoded timestamps, ensuring that keyframes within a specific time period can be traced. The "scene category horizontal axis" refers to classifying core keyframes into scenes (such as "classroom," "corridor," "playground," etc.) using image recognition algorithms, and assigning auxiliary keyframes to their corresponding scene categories. Each node in the index tree corresponds to a set of keyframes within a specific time period under a scene category.

[0075] Based on the same inventive concept, such as Figure 3 As shown, the present invention provides 8. a hybrid encoding processing system based on a video engine, comprising:

[0076] The video engine loading module 201 is used to start the video engine and synchronously load the H.265 main stream encoder and the MJPEG sub-stream encoder. This includes: the H.265 main stream encoder is enabled by default and configured to match the corresponding pixel block in the reference frame within a preset search range for each pixel block of the encoded frame, calculate the motion vector between pixel blocks, and then perform compensation processing on the pixel blocks based on the motion vector to eliminate inter-frame redundancy; the MJPEG sub-stream encoder preloads the JPEG quantization table and keeps it in standby mode.

[0077] Initialization module 202 is used to initialize a preset image recognition algorithm, including setting a first threshold for the proportion of the moving area and a threshold for the average absolute difference of pixels;

[0078] The motion region area ratio determination module 203 is used to calculate the motion region area ratio based on the initialized image recognition algorithm through edge detection and optical flow. If the motion region area ratio is lower than the first threshold, the MJPEG sub-stream encoder is triggered to compress the spatial redundancy data of a single frame using discrete cosine transform, and the timestamp index is recorded synchronously during encoding switching. If the motion region area ratio exceeds the first threshold, an H.265 encoding instruction is sent to enable inter-frame prediction to calculate the residual between the current pixel unit and the prediction unit in the current frame.

[0079] The non-repeating frame extraction module 204 is used to calculate the average absolute difference of pixels at corresponding positions in adjacent static frames. If the average absolute difference is lower than the average absolute difference threshold and the residuals in the same timestamp interval are all less than the average absolute difference threshold, it is marked as a repeating frame. The perceptual hash value of each frame is generated, and the Hamming distance of the perceptual hash values ​​is compared to confirm the key frame. If the key frame satisfies that the Hamming distance with the preceding and following frames exceeds the second threshold, the non-repeating frame with a hash difference exceeding the second threshold is retained, and the remaining frames are discarded.

[0080] The hybrid bitstream generation module 205 is used to filter the H.265 encoded video stream, write the H.265 encoded data corresponding to non-repeating frames into the video database according to the GOP structure, and store key frames and their corresponding perceptual hash values ​​into the image storage pool. During storage, a bidirectional correlation index is established by combining the timestamp index and the timestamps corresponding to each non-repeating frame. During decoding, a hybrid bitstream containing the H.265 main bitstream and the MJPEG sub-bitstream is reconstructed based on the bidirectional correlation index.

[0081] Based on the same inventive concept, such as Figure 4 As shown, the present invention provides an electronic device, including: a memory 302, a processor 301, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, it implements a hybrid encoding processing method based on a video engine.

[0082] Based on the same inventive concept, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a hybrid encoding processing method based on a video engine.

[0083] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A hybrid encoding processing method based on a video engine, characterized in that, The method comprises the following steps: Starting a video engine to synchronously load an H.265 main stream encoder and an MJPEG sub-stream encoder, including: the H.265 main stream encoder is started by default and is configured to match a corresponding pixel block in a reference frame within a preset search range for each pixel block of an encoded frame, calculate a motion vector between the pixel blocks, and perform compensation processing on the pixel blocks based on the motion vector to eliminate inter-frame redundancy; the MJPEG sub-stream encoder is preloaded with a JPEG quantization table and remains in a standby state; Initializing a preset image recognition algorithm, including setting a first threshold of a motion area proportion and a threshold of an average absolute difference of pixels; Based on the initialized image recognition algorithm, calculating the motion area proportion by edge detection and optical flow method; if the motion area proportion is lower than the first threshold, triggering the MJPEG sub-stream encoder to compress single-frame spatial redundant data by discrete cosine transform, and synchronously recording a timestamp index at the time of encoding switching; if the motion area proportion exceeds the first threshold, sending an H.265 encoding instruction to enable inter-frame prediction to calculate a residual error between a current pixel unit and a prediction unit in the current frame; Calculating an average absolute difference of pixels at corresponding positions in adjacent static picture frames, if the average absolute difference is lower than the threshold of the average absolute difference, and the residual error in the same timestamp interval is smaller than the threshold of the average absolute difference, marking as a repeated frame; generating a perceptual hash value of each frame, comparing the Hamming distance of the perceptual hash values to confirm a key frame, the key frame satisfying that the Hamming distance with the previous and subsequent frames exceeds a second threshold, retaining a non-repeated frame with a hash difference exceeding the second threshold, and discarding the remaining frames; Screening an H.265 encoded video stream, writing H.265 encoded data corresponding to the non-repeated frame into a video database according to a GOP structure, and storing the key frame and the corresponding perceptual hash value into an image storage pool; during storage, establishing a bidirectional association index in combination with the timestamp index and the timestamp corresponding to each non-repeated frame; during decoding, reconstructing a hybrid stream containing an H.265 main stream and an MJPEG sub-stream based on the bidirectional association index.

2. The hybrid encoding processing method based on a video engine according to claim 1, wherein, The preset search range of the H.265 main stream encoder is configured to be dynamically adjustable, including: identifying a texture-rich area and a smooth area in the picture; the texture-rich area is an area in which the frequency of brightness value change of adjacent pixels exceeds a majority proportion and the change amplitude exceeds a third threshold in a preset size of an image block; the smooth area is an area in which the frequency of brightness value change of adjacent pixels is lower than a preset proportion and the change amplitude is lower than a fourth threshold in a preset size of an image block; the search range is expanded to a first preset range for the texture-rich area; and the search range is reduced to a smaller coverage area of a second preset range for the smooth area.

3. The hybrid encoding processing method based on video engine according to claim 1, wherein, The step of synchronously recording a timestamp index at the time of encoding switching specifically comprises: Simultaneously recording timestamp information of the main stream and the sub-stream to form a double timestamp mapping table at the time of encoding mode switching; redundantly backing up the timestamp by using a local cache and an engine memory dual storage mechanism; in the subsequent decoding stage, ensuring the time continuity at the stream switching place by comparing the double timestamp mapping table.

4. The hybrid encoding processing method based on video engine according to claim 1, characterized in that, The step of enabling inter-frame prediction to calculate the residual between the current pixel unit and the prediction unit in the current frame specifically comprises: determining a reference frame set of the current encoding frame, the reference frame set containing at least one frame before the current encoding frame, at least one frame after the current encoding frame, or a combination thereof; dividing the current encoding frame into a plurality of pixel units, for each pixel unit, performing a matching search in the corresponding reference frame to obtain a reference pixel unit matched with the pixel unit; generating a prediction unit of the current pixel unit based on the reference pixel unit; and calculating the residual between the current pixel unit and the prediction unit.

5. The hybrid encoding processing method based on video engine according to claim 1, characterized in that, After the step of enabling inter-frame prediction to calculate the residual between the current pixel unit and the prediction unit in the current frame, the method further comprises: an initial motion vector determined by the position difference between the pixel unit and the matched reference pixel unit when performing the matching search in the corresponding reference frame for each pixel unit; smoothing the initial motion vector, removing an abnormal motion vector deviating from a preset angle of an overall trend based on the initial motion vector of the current pixel block and the initial motion vectors of adjacent pixel blocks, and replacing the abnormal motion vector with an average trend of the initial motion vectors of the adjacent pixel blocks; if the motion vector of the current pixel block and the motion vectors of the adjacent pixel blocks are not continuous in variation, correcting the motion vector of the current pixel block according to the variation trend of the motion vectors of the adjacent pixel blocks to keep the motion vectors of the adjacent pixel blocks continuous.

6. The hybrid encoding processing method based on video engine according to claim 1, wherein, The formula for calculating the average absolute difference of the pixels at the corresponding positions in the adjacent static picture frames is: wherein MAD is the mean absolute difference; M is the height of the pixel block; N is the width of the pixel block; i is a row index within the pixel block; j is a column index within the pixel block; I a (i,j) is the luminance or grayscale value of the pixel in the previous frame at row i and column j; I b (i,j) is the luminance or grayscale value of the pixel in the next frame at row i and column j.

7. The hybrid encoding processing method based on video engine according to claim 1, characterized in that, storing the key frames into an image storage pool, specifically comprising: dividing the key frames into core key frames and auxiliary key frames according to importance, the core key frames being video frames at scene switching, and the auxiliary key frames being action frames within a scene; the core key frames are stored by lossless compression, and the auxiliary key frames are stored by lossy compression; a key frame index tree is established to realize fast retrieval, with time sequence as the vertical axis and scene category as the horizontal axis.

8. A hybrid encoding processing system based on a video engine, characterized by, including: a video engine loading module for starting the video engine to synchronously load an H.265 main code stream encoder and an MJPEG sub-code stream encoder, including: the H.265 main code stream encoder being started by default and configured to match the corresponding pixel blocks in the reference frames within a preset search range for each pixel block of the encoding frame, calculate the motion vectors between the pixel blocks, and then perform compensation processing on the pixel blocks based on the motion vectors to eliminate inter-frame redundancy; the MJPEG sub-code stream encoder being preloaded with a JPEG quantization table and kept in a standby state; an initialization module for initializing a preset image recognition algorithm, including setting a first threshold of a motion area area ratio and a threshold of the average absolute difference of pixels; The motion area area ratio determination module is configured to calculate the motion area area ratio by edge detection and optical flow method according to the initialized image recognition algorithm; if the motion area area ratio is lower than a first threshold value, the MJPEG sub-code stream encoder is triggered to compress single-frame spatial redundant data by discrete cosine transform, and a time stamp index is recorded synchronously during encoding switching; if the motion area area ratio exceeds the first threshold value, an H.265 encoding instruction is sent to enable inter-frame prediction to calculate the residual error between the current pixel unit and the prediction unit in the current frame; The non-repeated frame extraction module is configured to calculate the average absolute difference of pixels at corresponding positions in adjacent static picture frames, mark a frame as a repeated frame if the average absolute difference is lower than the average absolute difference threshold value and the residual error in the same time stamp interval is smaller than the average absolute difference threshold value, generate a perceptual hash value of each frame, compare the Hamming distance of the perceptual hash values to confirm a key frame, and retain a non-repeated frame whose hash difference exceeds a second threshold value, and discard the remaining frames. The hybrid code stream generation module is configured to filter the H.265 encoded video stream, write the H.265 encoded data corresponding to the non-repeated frames into a video database according to the GOP structure, and store the key frames and the corresponding perceptual hash values in an image storage pool; during storage, a bidirectional association index is established in combination with the time stamp index and the time stamps corresponding to each non-repeated frame; and during decoding, the hybrid code stream containing the H.265 main code stream and the MJPEG sub-code stream is reconstructed based on the bidirectional association index.

9. An electronic device, comprising: The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the hybrid encoding processing method based on the video engine according to any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the hybrid encoding processing method based on the video engine according to any one of claims 1-7.

Images