Zero-copy video frame extraction system and method for hybrid development architecture

Abstract

The invention relates to the technical field of household breathing machines, in particular to a zero-copy video frame extraction system and method for a hybrid development architecture. The system comprises a video stream acquisition and decoding module used for receiving a video stream and decoding the video stream into a video frame; the zero-copy caching module is used for caching the video frames and writing the video frames into the annular buffer area in a zero-copy mode so as to avoid repeated memory copying; the frame extraction and analysis module is used for extracting the video frame from the annular buffer area, executing mask wearing state analysis and judging whether the video frame is a key frame or not; the key frame compression and storage module is used for compressing the key frame into a preset file format and storing a compressed file path; and the result processing module is used for returning an analysis result corresponding to the key frame and the compressed file path to the mixed framework layer and uploading the analysis result and the compressed file path to the server, and the result processing module replaces returning of the key frame by returning the compressed file path, so that the data transmission quantity is reduced, and additional memory copying is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of home ventilators, in particular to a system and method for zero-copy video frame extraction in a hybrid development framework. BACKGROUND

[0002] In the clinical application of home ventilators, the sealing performance of the user's mask is crucial: improper wearing can easily lead to gas leakage. To solve this problem, the existing technology usually uses video acquisition and analysis means to monitor the user's mask wearing state in real time. However, the current video processing scheme based on a hybrid development framework (such as UniApp hybrid development framework, Flutter hybrid development framework, ReactNative hybrid development framework, etc.) has many technical defects in actual application, and it is difficult to meet the requirements of real-time performance and stability in medical scenarios.

[0003] In the prior art, the video acquisition and analysis scheme based on RTSP (Real-Time Streaming Protocol) usually receives video streams by a player and decodes them, and then stores the decoded video frames in an in-process cache for subsequent call processing. Since the frame data is copied multiple times between the player, the cache and the analysis module, it not only causes a large memory copy overhead of the system and increases the energy consumption, but also easily causes memory overflow and application crash, and more importantly, it causes a significant time difference between the captured video frames and the actual mask wearing action of the user, directly affecting the accuracy of key frame extraction and greatly weakening the real-time processing capability of the system. In addition, the existing implementation lacks a unified frame stream interface and an efficient cache management strategy, and has poor extensibility and cross-platform adaptability, which not only makes it difficult to meet the strict requirements of performance and resource efficiency in real-time detection scenarios for home ventilators, but also significantly increases the maintenance cost and upgrade difficulty of the system. SUMMARY

[0004] The purpose of the present application is to overcome the problems of high video delay, large memory occupation and poor extensibility in the detection of mask wearing of a home ventilator, and to provide a system and method for zero-copy video frame extraction in a hybrid development framework to improve the real-time performance, stability and system maintainability of the detection.

[0005] To solve the above technical problems, the system for zero-copy video frame extraction in a hybrid development framework provided by the technical solution of the present application comprises: a video stream acquisition and decoding module for receiving video streams and decoding them into video frames; a zero-copy cache module for caching the video frames and writing them into a ring buffer in a zero-copy manner to avoid repeated memory copying; a frame extraction and analysis module for extracting video frames from the ring buffer and performing mask wearing state analysis, and determining whether the extracted video frames are key frames based on the analysis results. A keyframe compression and storage module is used to compress the keyframes into a predetermined file format and store the compressed file path; and, The results processing module is used to send the analysis results and compressed file paths corresponding to the keyframes back to the hybrid framework layer and upload them to the server. The results processing module sends back the compressed file paths instead of sending back the keyframes themselves, thereby reducing the amount of data transmission and avoiding additional memory copying.

[0006] As an improvement to the above system, the circular buffer is used to cache video frames with a fixed capacity; when the circular buffer is full, a cyclic overwrite strategy or a trigger processing strategy is adopted to balance real-time performance and memory usage.

[0007] As an improvement to the above system, the cyclic overwrite strategy overwrites the earliest stored video frame, and the trigger processing strategy releases the space immediately after processing the cached video frame.

[0008] As an improvement to the above system, the video stream acquisition and decoding module, the zero-copy buffer module, and the frame extraction and analysis module share video frames through pointers or memory mapping to achieve zero copy.

[0009] As an improvement to the above system, the video stream acquisition and decoding module and the zero-copy buffer module are encapsulated in the EasyRTSP framework, and the frame extraction and analysis module is encapsulated in the SelecterRTSP framework.

[0010] As an improvement to the above system, the video stream acquisition and decoding module supports both software and hardware decoding by integrating a decoding stack consisting of EasyPlayer and FFmpeg.

[0011] As an improvement to the above system, the frame extraction and analysis module extracts video frames according to a frame skipping strategy and integrates a mask analysis algorithm based on computer vision to perform mask fit analysis; the frame extraction and analysis module determines whether the extracted video frames are key frames based on the fit analysis results, and generates potential air leakage risk labels for key frames.

[0012] As an improvement to the above system, the keyframe compression and storage module starts a compression thread, compresses the keyframes into a predetermined file format, and writes them to the system temporary directory; the keyframe compression and storage module saves the path of the compressed file to a thread-safe array; the result processing module collects the compressed file paths corresponding to the keyframes stored in the thread-safe array, aligns the compressed file paths with the analysis results corresponding to the keyframes, generates a complete result dataset, sends it back to the hybrid framework layer, and uploads it to the server.

[0013] As an improvement to the above system, the compressed file format includes: JPEG format; the video frame includes NV21 format video frame; the hybrid framework layer includes UniApp, Flutter and ReactNative frameworks, and interacts with native modules through plugins or bridging mechanisms.

[0014] To achieve another objective of the present invention, the present invention also provides a zero-copy video frame extraction method for hybrid development architectures, implemented based on the above-described zero-copy video frame extraction system for hybrid development architectures, comprising: Step 1: Receive the video stream and decode it into video frames through the video stream acquisition and decoding module; Step 2: Cache the video frames using a zero-copy caching module and write the video frames into a circular buffer in a zero-copy manner to avoid repeated memory copies; Step 3: Extract video frames from the circular buffer using the frame extraction and analysis module and perform mask wearing status analysis. Determine whether the extracted video frames are key frames based on the analysis results. Step 4: Compress the keyframes into a predetermined file format using the keyframe compression and storage module, and store the compressed file path; Step 5: The analysis results and compressed file path corresponding to the keyframes are sent back to the hybrid framework layer and uploaded to the server through the result processing module. The result processing module sends back the compressed file path instead of sending back the keyframes themselves, reducing the amount of data transmission and avoiding additional memory copying.

[0015] The advantages of this application are: 1. Reduce video playback latency in hybrid development frameworks such as UniApp to ensure that the extracted video frames are synchronized with the user's mask-wearing actions; 2. By performing zero-copy processing on video frames, the CPU usage and memory overhead caused by repeated data copying are reduced; 3. Implement real-time compression and temporary storage of keyframes to avoid excessive memory spikes and adapt to resource-constrained mobile scenarios; 4. Adopting a plug-in architecture design, it provides a unified cross-platform interface, improving the system's scalability and maintainability. Attached Figure Description

[0016] Figure 1 The flowchart illustrates the workflow of the zero-copy video frame extraction system for hybrid development architecture provided by this invention. Detailed Implementation

[0017] The technical solution of this invention includes two parts: system architecture design and method implementation. Through modular encapsulation, zero-copy data transmission, and real-time compression and peak control, it achieves efficient video frame extraction and mask wearing detection.

[0018] The following is combined with Figure 1 The present invention will be further described in detail with reference to specific embodiments. The figures include the following key identifiers: Hybrid architecture layer, which includes hybrid development frameworks such as UniApp, Flutter, and React Native; The video stream acquisition and decoding module, namely EasyRTSP.framework; The frame extraction and analysis module, namely SelecterRTSP.framework; The process nodes include "Start Playback", "Set Parameters", "Start Recording", "Frame Extraction Analysis", "JPEG Compression", "End Recording", "End Playback", "Result Callback", and "Upload Result". The data flow is as follows: the RTSP video stream first enters the video stream acquisition and decoding module, and after decoding, the NV21 frames are output and then passed to the circular buffer (NV21RingBuffer). The frame extraction and analysis module then extracts and analyzes the data, and then it enters the JPEG compression stage. After compression, the result is sent back to the hybrid architecture layer, and finally the hybrid architecture layer uploads the result to the server.

[0019] (I) System Architecture and Data Flow This system mainly consists of five core modules, which are connected through standardized interfaces, ensuring clear and controllable data flow. Specific information about each module is as follows: 1. Video Stream Acquisition and Decoding Module: This module is encapsulated as EasyRTSP.framework. Its function is to receive, decode, and play RTSP video streams, and output NV21 format video frames. Its core technical features are: integrating EasyPlayer player and FFmpeg into a software decoding stack, while also supporting hardware decoding, outputting NV21 format video frames, and providing a read-only memory view of continuous frames.

[0020] 2. Zero-Copy Buffer Module: This module, encapsulated as EasyRTSP.framework, implements zero-copy shared storage of video frames, ensuring efficient reading and processing of decoded frames. It coordinates data interaction between the decoding thread (producer) and the frame extraction thread (consumer). Its core technical features include: establishing an NV21RingBuffer (NV21 frame circular buffer) with a fixed capacity (e.g., a maximum of 1200 frames), sharing decoded frames via pointers or memory mapping, eliminating secondary copying. The decoding thread (producer) writes data, and the frame extraction thread (consumer) reads data. When the buffer is full, it supports two full-load strategies: "circular overwrite" (overwriting the earliest stored frame) or "triggered processing" (immediately processing the buffered frame and then releasing the space), balancing real-time performance and memory usage, ensuring real-time performance while controlling peak memory usage.

[0021] 3. Frame Extraction and Analysis Module: This module is encapsulated as SelecterRTSP.framework. Its function is to extract NV21 frames according to a frame skipping strategy and perform mask fit analysis and leakage risk assessment. Its core technical features are: integrating mask region detection and fit assessment within SelecterRTSP.framework, which can employ detection algorithms based on facial keypoint localization + region segmentation / optical flow / edge sealing indicators. It outputs fit score, offset, and potential leakage risk labels, and binds them to the corresponding keyframe paths.

[0022] 4. Keyframe Compression and Storage Module: This module compresses extracted keyframes in JPEG format and stores the compressed file in the system's temporary directory, saving only the file path rather than the complete frame data. Its core technical features are: once a keyframe is selected, the compression operation is immediately performed through a background queue; the JPEG compression quality is configurable (e.g., set to 0.7); and a thread-safe array is used to store the file path to avoid data corruption.

[0023] 5. Results Processing Module (Hybrid Framework Layer): Its function is to send the keyframe file path, mask fit score, and potential air leakage risk label back to the hybrid framework (such as UniApp), and support uploading the results to a remote server; its core technical features are: providing a unified callback interface, supporting data transmission in JSON (JavaScript Object Notation) or dictionary format, and ensuring compatibility with the hybrid framework.

[0024] The data flow path of this system is: RTSP stream → decoding output → circular buffer → frame extraction analysis → JPEG compression → path aggregation → callback to the hybrid framework layer → upload to the server. Specifically: the RTSP video stream first enters the video stream acquisition and decoding module, and after decoding, outputs NV21 frames and writes them to the circular buffer; the frame data in the circular buffer is extracted by the frame extraction and analysis module and analyzed; the frame data determined to be critical enters the key frame compression and storage module, is JPEG compressed and stored in a temporary directory with the path recorded; the result processing module aggregates the key frame paths and detection results, calls back to the hybrid framework layer, and finally the hybrid framework layer uploads the results to the remote server.

[0025] (II) Detailed Implementation of Core Modules 1. Video Stream Acquisition and Decoding Module: Decoding and Circular Buffer Design (1) Decoding kernel: It integrates a decoding stack composed of EasyPlayer and FFmpeg, and supports both hardware decoding and software decoding. After decoding, it directly outputs NV21 format video frames, provides a read-only memory view of continuous frames, avoids the extra overhead caused by subsequent format conversion, and reduces playback latency.

[0026] (2) Circular Buffer (NV21RingBuffer): A circular buffer with a fixed capacity is used. The maximum number of frames is specified by the "maximum number of frames parameter" (e.g., 1200 frames). Decoded video frames are shared through pointers or memory mapping. There is no secondary copy operation, which improves the efficiency of frame extraction and analysis and controls memory peak. The decoding thread acts as the producer to write NV21 frames into the buffer, and the frame extraction thread acts as the consumer to read frame data from the buffer. When the buffer is full, the real-time performance and memory usage are balanced through the "circular overwrite" or "trigger processing" strategy.

[0027] (3) Frame preprocessing: Lightweight conversion of different format video frames into NV21 format frames; the conversion result is directly written to the circular buffer, and only one memory write operation is performed, so that the frame extraction and analysis module can read it in a copy-free manner.

[0028] 2. Real-time compression of keyframes and memory safety control When a video frame is identified as a keyframe (i.e., meets the feature requirements for mask detection), the system immediately starts a background thread to perform the following operations: (1) JPEG compression: Compress the keyframes in NV21 format into JPEG format. The compression quality is controlled by the "JPEG quality parameter", and the parameter value range is 0.2 to 0.9, for example, set to 0.7.

[0029] (2) Temporary storage: Write the compressed JPEG file to the system temporary directory.

[0030] (3) Memory optimization: Only the path of JPEG files is saved to a thread-safe array, and large video frame objects are not retained; at the same time, the automatic release pool mechanism is combined to limit the peak memory, ensure the stability of mobile resources-constrained environments, and also avoid application crashes on the iOS platform.

[0031] 3. Plug-in encapsulation and cross-hybrid framework interface design To adapt to hybrid development frameworks such as UniApp, Flutter, and ReactNative, this system adopts modular encapsulation and a unified interface design, as detailed below: (1) Modular encapsulation: The video stream acquisition and decoding module and the circular buffer are encapsulated as EasyRTSP.framework, and the frame extraction and analysis function is encapsulated as SelecterRTSP.framework; the two frameworks achieve zero-copy data interaction across components through shared memory or pointers to avoid repeated data transmission.

[0032] (2) Unified cross-hybrid framework interface: Native modules expose core interfaces such as "RTSP start playback", "RTSP stop playback" and "RTSP start recording" to hybrid frameworks (UniApp, Flutter, ReactNative, etc.) through Plugin or Bridge mechanisms. Among them, "RTSP start recording" requires the recording mode and whether to enable frame skipping parameters, and supports dynamically configured decoding methods, frame skipping switch, interval and compression quality; the script layer of the hybrid framework dynamically passes parameters, the native layer responds in real time, and returns the path of the filtered keyframe file and operation status identifier.

[0033] (3) Thread model and cross-bridge data security: The system adopts multi-threaded collaborative processing, and the thread flow is decoding thread → circular buffer → frame extraction processing thread → JPEG compression thread → main thread (UI (user interface) distribution); NV21 format data is transmitted through zero-copy pointer or memory mapping to avoid performance bottlenecks caused by cross-frame transmission of large objects; lightweight data such as key frame paths and fit scores are returned to the hybrid framework in JSON or dictionary format to ensure the security and efficiency of data transmission.

[0034] 4. Frame extraction algorithm and mask wearing detection implementation The core functions of the frame extraction and analysis module include frame extraction logic and mask wearing detection, as detailed below: (1) Frame extraction interface design: Define a standardized frame extraction interface, which mainly includes three types of functional interfaces: initialization configuration, frame extraction mode setting and frame data reception, respectively implementing core functions such as parameter configuration, mode selection, data reading and starting frame extraction analysis.

[0035] (2) Frame extraction logic: Candidate frames are extracted from the circular buffer according to the frame skipping strategy through a preset loop processing function. The specific process is as follows: First, the NV21 frames in the circular buffer are read; then, candidate frames are extracted according to the preset frame extraction sampling interval; after processing the candidate frames, the relevant resources are released; when the number of frames in the buffer reaches the maximum capacity or the video playback ends, the frame extraction loop is exited; finally, the mapping result of the key frame is output (such as the correspondence between the frame number and the corresponding file path).

[0036] (3) Mask wearing detection: A computer vision-based mask analysis algorithm is integrated into SelecterRTSP.framework to perform mask fit analysis. Based on the fit analysis results, it is determined whether the extracted video frame is a key frame. If it is determined to be a key frame, a potential air leakage risk label (such as "no air leakage", "slight air leakage", "serious air leakage") is generated. The fit analysis results, air leakage risk labels and the compressed file path of the corresponding key frame are bound together for use by the result processing module.

[0037] (III) Implementation process of the method steps The zero-copy video frame extraction method of the present invention is based on the above system architecture and specifically includes the following seven steps: 1. Initialization and Video Playback Startup: Set video processing parameters in the UniApp and other hybrid framework layers, including software decryption switch (isFFMpeg), frame sampling interval, and JPEG quality, and then call the "RTSP Start Playback" interface; The video stream acquisition and decoding module (EasyRTSP.framework) receives the RTSP video stream, starts decoding (software decoding or hardware decoding), outputs NV21 format frames, and writes the frame data to the circular buffer (NV21RingBuffer).

[0038] 2. Recording and frame extraction process start: Call the “RTSP Start Recording” interface, and the system starts the “Frame Extraction Loop Processing Function (processNV21Loop)” to extract candidate frames from the circular buffer according to the preset frame extraction sampling interval.

[0039] 3. Zero-copy frame extraction and mask analysis: The frame extraction and analysis module reads candidate frames from the circular buffer in a copy-free manner through pointers or memory mapping; it performs mask wearing detection on the candidate frames, including facial key point localization, mask region segmentation, and fit evaluation, to determine whether the candidate frame is a key frame.

[0040] 4. Real-time compression and storage of keyframes: If a candidate frame is determined to be a keyframe, the system starts a background compression thread to compress the NV21 frame into JPEG format at a preset JPEG quality and write it to the system temporary directory (such as NSTemporaryDirectory() in iOS system); only the path of the JPEG file is saved to a thread-safe array, and the memory occupied by the frame data is released.

[0041] 5. Cache memory peak control: Real-time monitoring of the number of available frames in the circular buffer. When the number of available frames reaches a preset threshold, or the number of frames in the buffer reaches the "maximum number of frames parameter" (e.g., 1200 frames), the system prioritizes processing and releasing the earliest stored frame data to ensure that memory usage remains stable within a controllable range.

[0042] 6. Process Closure and Result Summary: When the video playback ends or the "RTSP Stop Recording" interface is triggered, the system waits for the frame extraction processing thread and the compression thread to complete the remaining tasks; it collects the keyframe file paths stored in the thread-safe array, aligns them with the mask detection results (fit score, air leakage risk label), and generates a complete result dataset.

[0043] 7. Result Feedback and Upload: The filtered keyframe file path and operation status identifier are fed back to the hybrid frame layer through the "frame extraction result callback interface"; after receiving the results, the hybrid frame layer uploads the keyframe file and detection results to the remote server to complete the entire process of mask wearing detection.

[0044] Compared with the prior art, the present invention has the following significant advantages: 1. This application significantly reduces data latency and memory overhead through a zero-copy processing mechanism using a circular buffer. Specifically, the circular buffer relies on pointer or memory mapping technology to achieve cross-component data sharing with the video stream acquisition and decoding module and the frame extraction analysis module. There are no secondary copy operations throughout the process, which can significantly reduce CPU usage while improving frame extraction analysis efficiency and effectively controlling memory peaks. At the same time, the decoding thread writes data directly to the buffer as a "producer," and the frame extraction thread reads data directly as a "consumer." Combined with the full-load strategy of "circular overwrite" or "triggered processing," a dynamic balance between real-time performance and memory usage is achieved.

[0045] 2. In response to the resource constraints of mobile devices, this application adopts a "real-time compression + lightweight storage" strategy to process keyframes: when a video frame is determined to be a keyframe, the system immediately starts a background thread to compress it into a lightweight JPEG format; after compression, only the file path is stored in a thread-safe array, and the large video frame object is not retained; at the same time, combined with the "circular overwrite" or "triggered processing" full-load strategy of the circular buffer, the memory peak is strictly controlled, effectively avoiding the application crash problem caused by memory overload on mobile devices, and adapting to the continuous running requirements of mobile devices.

[0046] 3. Through flexible frame extraction strategies and dynamic cache management, this application achieves a synergistic balance between real-time performance and system stability: At the frame extraction strategy level, a configurable frame skipping mechanism is designed. The frame extraction analysis module extracts video frames according to a preset frame skipping strategy (e.g., extracting 1 frame every 3 frames), reducing resource waste caused by invalid frame processing; At the cache management level, when the number of available frames in the circular buffer reaches a preset threshold, or when the number of frames in the buffer reaches its maximum capacity, the system prioritizes processing and releasing the earliest stored frame data to prevent the risk of buffer overflow; At the same time, combined with the key frame path storage mechanism, a lightweight file path is used to replace the complete frame data residing in memory, further reducing memory load and achieving precise control of memory usage.

[0047] 4. Reduce video latency and ensure detection synchronization: By integrating EasyPlayer and FFmpeg into a software / hardware decoding stack, NV21 format frames are directly output, reducing the format conversion process; combined with zero-copy data sharing of the NV21 RingBuffer, frequent data interaction delays between the script layer and the native layer are avoided, making the frame extraction time highly synchronized with the user's mask wearing action, thus improving detection accuracy.

[0048] 5. Plug-in architecture with strong cross-platform adaptability: Core functions are encapsulated into EasyRTSP.framework and SelecterRTSP.framework respectively, and are adapted to hybrid frameworks such as UniApp, Flutter, and ReactNative through a unified interface; a single codebase can run on multiple platforms such as iOS and Android, reducing cross-platform development workload, lowering maintenance costs, and improving system scalability.

[0049] 6. Multi-threaded collaboration improves processing efficiency: The system adopts a multi-threaded model of "decoding → caching → frame extraction → compression → UI distribution" to process each stage in parallel. Data is passed through pointers or memory mapping to avoid performance bottlenecks caused by cross-frame transmission of large objects. The overall processing efficiency is improved by more than 40% compared with traditional solutions, meeting the real-time requirements of home ventilator mask detection.

[0050] 7. Standardized frame extraction interface with excellent ease of use and scalability: The standardized frame extraction interface is defined in SelecterRTSP.framework. The interface parameters are clear and the functions are modular, which makes it easy to replace the frame extraction algorithm or extend to a new frame extraction mode without modifying the overall architecture, thus improving the system maintainability.

[0051] 8. Intelligent detection, adapting to medical scenario needs: It integrates mask fit assessment and leakage risk analysis functions, outputs quantitative scores and risk labels, not only to realize basic wearing status judgment, but also to provide medical staff with accurate risk warning information, thereby enhancing the medical application value of home ventilators.

[0052] In summary, this invention, through its overall technical solution of "zero-copy ring buffer (NV21RingBuffer) + real-time keyframe compression (JPEG) + pluggable cross-platform interface (Plugin / Bridge)," comprehensively solves the problems of high latency, large memory consumption, and poor scalability in existing technologies, and significantly improves the real-time performance, stability, intelligence level, and scalability of home ventilator mask wearing detection.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A zero-copy video frame extraction system for hybrid development architectures, characterized in that, include: The video stream acquisition and decoding module is used to receive video streams and decode them into video frames; The zero-copy buffer module is used to buffer the video frames and write the video frames into a circular buffer in a zero-copy manner to avoid repeated memory copies. The frame extraction and analysis module is used to extract video frames from the circular buffer and perform mask wearing status analysis, and determine whether the extracted video frames are key frames based on the analysis results. A keyframe compression and storage module is used to compress the keyframes into a predetermined file format and store the compressed file path; and, The results processing module is used to send the analysis results and compressed file paths corresponding to the keyframes back to the hybrid framework layer and upload them to the server. The results processing module sends back the compressed file paths instead of sending back the keyframes themselves, thereby reducing the amount of data transmission and avoiding additional memory copying.

2. The zero-copy video frame extraction system for hybrid development architecture according to claim 1, characterized in that, The circular buffer is used to cache video frames with a fixed capacity; when the circular buffer is full, a cyclic overwrite strategy or a trigger processing strategy is adopted to balance real-time performance and memory usage.

3. The zero-copy video frame extraction system for hybrid development architecture according to claim 2, characterized in that, The cyclic overwrite strategy overwrites the earliest stored video frame, and the trigger processing strategy releases the space immediately after processing the cached video frame.

4. The zero-copy video frame extraction system for hybrid development architecture according to claim 1, characterized in that, The video stream acquisition and decoding module, the zero-copy buffer module, and the frame extraction and analysis module share video frames through pointers or memory mapping to achieve zero copying.

5. The zero-copy video frame extraction system for hybrid development architecture according to claim 4, characterized in that, The video stream acquisition and decoding module and the zero-copy caching module are encapsulated in the EasyRTSP framework, and the frame extraction and analysis module is encapsulated in the SelecterRTSP framework.

6. The zero-copy video frame extraction system for hybrid development architecture according to claim 1, characterized in that, The video stream acquisition and decoding module supports both software and hardware decoding by integrating a decoding stack consisting of EasyPlayer and FFmpeg.

7. The zero-copy video frame extraction system for hybrid development architecture according to claim 1, characterized in that, The frame extraction and analysis module extracts video frames according to a frame skipping strategy and integrates a computer vision-based mask analysis algorithm to perform mask fit analysis. The frame extraction and analysis module determines whether the extracted video frames are key frames based on the fit analysis results and generates potential air leakage risk labels for key frames.

8. The zero-copy video frame extraction system for hybrid development architecture according to claim 1, characterized in that, The keyframe compression and storage module starts a compression thread, compresses the keyframes into a predetermined file format, and writes them to the system temporary directory. The keyframe compression and storage module saves the path of the compressed file to a thread-safe array; The result processing module collects the compressed file paths corresponding to the keyframes stored in the thread-safe array, aligns the compressed file paths with the analysis results corresponding to the keyframes, generates a complete result dataset, sends it back to the hybrid framework layer, and uploads it to the server.

9. The zero-copy video frame extraction system for hybrid development architecture according to claim 1, characterized in that, The compressed file format includes JPEG format; the video frame includes NV21 format video frame; the hybrid framework layer includes UniApp, Flutter and ReactNative frameworks, and interacts with native modules through plugins or bridging mechanisms.

10. A zero-copy video frame extraction method for hybrid development architectures, implemented based on the zero-copy video frame extraction system for hybrid development architectures as described in any one of claims 1-9, characterized in that, include: Step 1: Receive the video stream and decode it into video frames through the video stream acquisition and decoding module; Step 2: Cache the video frames using a zero-copy caching module and write the video frames into a circular buffer in a zero-copy manner to avoid repeated memory copies; Step 3: Extract video frames from the circular buffer using the frame extraction and analysis module and perform mask wearing status analysis. Determine whether the extracted video frames are key frames based on the analysis results. Step 4: Compress the keyframes into a predetermined file format using the keyframe compression and storage module, and store the compressed file path; Step 5: The analysis results and compressed file path corresponding to the keyframes are sent back to the hybrid framework layer and uploaded to the server through the result processing module. The result processing module sends back the compressed file path instead of sending back the keyframes themselves, which reduces the amount of data transmission and avoids additional memory copying.

Images