An image acquisition device and method based on a Flutter platform

Abstract

The application discloses an image acquisition method device and method based on a Flutter platform, and is used for solving the problems of large performance loss, high cross-platform cost and weak expansibility of an existing scheme. An image processing layer is additionally arranged between a Flutter Engine and a system display layer, and a switch control and an image callback interface are configured. When the switch is closed, the original drawing process is followed, and no additional cost is needed. When the switch is opened, a display layer frame buffer is associated, a new drawing canvas is added for drawing by the Flutter Engine, a drawing completion signal is listened to, image data is acquired, and the image data is returned and synchronously written into the frame buffer for normal display. The application realizes efficient acquisition by relying on the drawing performance of the Flutter, one set of code is cross-platform reused, extension such as streaming, analysis, recording and real-time communication is supported, and the application can be applied to different devices by relying on the cross-platform capability of the Flutter, and has high expandability and customization capability.

Description

Technical Field

[0001] This invention relates to the field of cross-platform graphical interface development technology, specifically to an image acquisition device and method based on the Flutter platform, applicable to screen capture, real-time video processing, image recording and content analysis scenarios in mobile, desktop and embedded Flutter applications. Background Technology

[0002] In the existing technologies for image acquisition on the Flutter platform, there are two main implementation schemes. Both of them have intractable technical defects and cannot meet the core requirements of Flutter cross-platform development for image acquisition that are "accurate, low-power, compatible, and easily scalable," as detailed below: 1. System-level screen recording and capture solution: This solution captures frame data from the entire device screen in real time by calling the device system's native screen recording interface. The core drawbacks of this solution are: ① Inaccurate target area capture: It simultaneously captures screen content from non-Flutter applications, making it impossible to accurately extract image data drawn solely by Flutter applications, requiring additional data cropping; ② Significant system performance loss: It continuously consumes core system resources such as CPU, memory, and graphics card, causing a significant drop in the frame rate of Flutter applications, resulting in lag and even application crashes and system freezes on low-configuration mobile devices; ③ High capture latency: The frame data processing chain for system-level screen recording is long, with a capture-to-display latency exceeding 100ms, which cannot meet the low-latency requirements of real-time video processing and real-time communication scenarios.

[0003] 2. Multi-Platform Native Development Acquisition Solution: This solution involves developing separate image acquisition modules based on the platform's native interfaces for different operating systems such as Android and iOS. Data interaction is achieved through a bridge between Flutter and the native interfaces. The core drawbacks of this solution are: ① Extremely high development and maintenance costs, requiring manpower for multi-platform native development, and significant differences in native interfaces across platforms, making interface adaptation cumbersome; ② Poor cross-platform compatibility, with independent development of acquisition logic for each platform, easily leading to inconsistencies in acquisition resolution, frame rate, and data format; ③ Weak functional scalability, requiring iterative development on multiple platforms when adding image processing capabilities such as streaming, image analysis, and local recording. This results in high coupling between acquisition logic and business logic, leading to low development iteration efficiency.

[0004] In summary, the common technical bottlenecks of existing technologies are: none of them are designed with a dedicated image acquisition solution based on the native Flutter architecture. Either they acquire images indirectly from the system layer, which leads to performance and accuracy issues, or they develop for multiple platforms without Flutter's cross-platform capabilities, which leads to cost and scalability issues. There is an urgent need for an image acquisition method based on Flutter's native rendering logic to solve the above technical pain points from the bottom layer. Summary of the Invention

[0005] Addressing common technical bottlenecks in existing technologies, this invention abandons the traditional system-level screen recording and multi-platform native development approach, proposing an image acquisition device and method based on the Flutter platform. Its core is the addition of a lightweight, non-intrusive, independent image processing layer between the Flutter Engine and the system display layer. Through a core design of "on-demand control + simultaneous dual-canvas acquisition + standardized interface adaptation," combined with Flutter's cross-platform capabilities, a single codebase for image acquisition is reused across multiple platforms. This design does not interfere with the Flutter Engine's native rendering logic and fundamentally solves the problems of high performance overhead, high cross-platform costs, and weak scalability in existing technologies, balancing image acquisition accuracy, performance stability, and functional extensibility.

[0006] The technical solution of the present invention is as follows: An image acquisition device based on the Flutter platform includes: The Flutter application layer is used to send start / stop commands for data collection, receive collected data, and connect to external extension modules. Flutter Engine is the core rendering engine of the Flutter platform, which performs native image rendering and sends a drawing completion signal; The image processing layer is a lightweight, independent layer added between the Flutter Engine and the system display layer to realize image data acquisition, transmission, synchronization, and control functions. The system display layer, the native display layer of the terminal device, is responsible for the final display of images, and different platforms adapt to different implementations. External extension modules are functional modules that connect to standardized callback interfaces, supporting functions including streaming, image analysis, recording, and real-time communication.

[0007] Furthermore, the image processing layer includes: The acquisition control module is responsible for the frame buffer association of the system display layer, the creation of independent synchronous canvases, and the execution of image acquisition operations; The switch control module, a Boolean control core, is used to receive API commands from the application layer and control the start and stop of the data acquisition function and the signal flow between modules. The data monitoring module listens for the Flutter Engine's drawing completion signal and triggers the acquisition control module to execute the acquisition job. The interface adaptation module enables standardized conversion of collected data formats and completes data return and connection with external extension modules through a unified callback interface.

[0008] This invention also provides an image acquisition method based on the Flutter platform, comprising the following steps: Step 1: Add a lightweight independent image processing layer between Flutter Engine and the system display layer. The image processing layer only contains acquisition control, data monitoring, and interface adaptation modules, and does not interfere with the native rendering logic of Flutter Engine, so as to realize image data acquisition, transmission and synchronization. Step 2: Configure a switch control module and a standardized image callback interface in the image processing layer. The switch control module controls the start and stop of the image acquisition function, and the image callback interface defines a unified data format and calling standard for image data return and external expansion module docking. Step 3: Execute the process according to the state of the switch control module: When closed, the native Flutter drawing process is used, and the image processing layer sleeps without additional overhead; when open, the system display layer frame buffer is associated, an independent drawing canvas synchronized with the native rendering canvas is added, and after listening for the Flutter Engine drawing completion signal, the image data in the canvas is collected. At the same time, the data is sent back through the image callback interface and synchronously written to the system display layer frame buffer to achieve normal display. Step 4: Leveraging Flutter's cross-platform capabilities, implement a single codebase for reusing the core image acquisition logic across multiple platforms. Connect to one or more extended modules from streaming, image analysis, recording, and real-time communication via the image callback interface to extend the image acquisition functionality.

[0009] Furthermore, in step 1, the system display layer is CALayer on iOS or SurfaceView on Android. The image processing layer establishes a standardized data stream connection with the frame buffer of the display layer of different platform systems to adapt to the frame buffer data format and read / write specifications of multiple platforms.

[0010] Furthermore, in step 2, the switch control module is a Boolean control logic module that supports dynamically switching between on and off states via the Flutter application layer API, enabling image acquisition to be enabled on demand.

[0011] Furthermore, in step 2, when the switch control module is turned on, the collected image data includes original pixel data, resolution, and rendering frame rate information, and the collected data is consistent with the data displayed in the system display layer, with no delay in the synchronization of collection and display.

[0012] Furthermore, in step 2, the independently drawn canvas is generated by associating with the texture ID and is synchronized in real time with the rendering rhythm and data content of the native rendering canvas, avoiding display abnormalities caused by directly manipulating the frame buffer.

[0013] Furthermore, in step 2, when the switch control module is turned off, the independent drawing canvas, the drawing completion signal listener and related acquisition threads are automatically destroyed, system resources are released and the Flutter native drawing process is restored.

[0014] The core innovation of this invention is: 1. Non-intrusive, lightweight image processing layer design: Unlike existing technologies that lack a dedicated Flutter capture layer, this design adds an independent image processing layer between the Flutter Engine and the system display layer. It contains only three core modules: capture control, data monitoring, and interface adaptation, with no redundant functions. This layer does not modify the Flutter Engine's native rendering logic or the system display layer architecture; it serves solely as a data transfer and capture carrier, resolving the issues of inaccurate capture targets and redundant data in non-target areas found in existing technologies.

[0015] 2. On-demand acquisition mechanism with controllable switching: A switch control module that supports dynamic switching is configured in the image processing layer. When turned off, the layer sleeps, completely following the native Flutter rendering process without any additional performance overhead; the acquisition process is triggered only when turned on. This invention differs from existing technologies that occupy system resources throughout the acquisition process, solving the problems of high performance loss and application lag in existing technologies.

[0016] 3. Dual-canvas design for synchronized data acquisition and display: When the switch is enabled, a new independent drawing canvas is added, which is synchronized in real time with the Flutter Engine's native rendering canvas. Data is acquired from this canvas and simultaneously written to the system's display layer frame buffer. This invention differs from existing technologies that separate the acquisition and display links, solving the problems of high acquisition latency and display anomalies caused by direct manipulation of the frame buffer in existing technologies.

[0017] 4. Standardized and Extensible Image Callback Interface: A unified data format and calling standard image callback interface is designed and defined, serving as a common data entry point for data acquisition, the Flutter application layer, and external extension modules. This invention differs from existing technologies with their multi-terminal independent interface designs, solving the problems of repetitive multi-terminal development required for extended functions and high coupling between data acquisition and business logic.

[0018] 5. Flutter Native Cross-Platform Reusability Design: This design achieves deep compatibility between the image processing layer and Flutter Engine's native rendering logic. The core data acquisition logic leverages Flutter's cross-platform capabilities to implement a single codebase adapted for mobile, desktop, and embedded Flutter applications. This invention differs from existing multi-platform native development methods, solving the problems of high development and maintenance costs and poor compatibility associated with current cross-platform technologies.

[0019] The present invention has the following beneficial effects: 1. Excellent capture performance with no additional resource consumption: It adopts a lightweight image processing layer design with no additional performance burden. When the switch is off, the performance is completely consistent with the native Flutter application. When it is on, the capture frame rate is synchronized with the Flutter rendering frame rate (up to 60fps). The system CPU and memory usage is reduced by more than 40% compared with the existing system layer screen recording solutions, and there are no issues with application lag or crashes.

[0020] 2. Accurate and synchronous data collection with no delay or anomalies: Only image content drawn by the Flutter application is collected, with no data from non-target areas. The collected data is 100% consistent with the display data of the system display layer. The latency between collection and display is less than 20ms, and it avoids display anomalies such as screen tearing and frame skipping caused by directly manipulating the frame buffer, meeting the needs of low-latency scenarios.

[0021] 3. Extremely low cross-platform cost and strong adaptability: The core logic of data collection is adapted to Android and iOS mobile terminals, Windows and Mac desktop terminals and embedded Flutter applications. No need for multi-platform native development, cross-platform development cost is reduced by more than 80%, and subsequent maintenance efficiency is improved by 90%.

[0022] 4. High scalability and low coupling: The standardized image callback interface supports flexible integration with extended modules such as streaming, image analysis, local recording, and real-time communication. New extended functions only need to be developed once, without the need for multi-terminal adaptation, achieving complete decoupling between acquisition logic and business logic.

[0023] 5. Non-intrusive and easy to integrate, with low transformation cost: It does not modify the Flutter native architecture and the underlying implementation of the system display layer, and can be directly integrated into various existing Flutter applications. The integration process does not require modification of the original business code, resulting in low transformation cost and wide compatibility. 6. Achieved technological breakthroughs and has high application value: It has achieved technological breakthroughs in image acquisition on the Flutter platform, combining creativity, practicality and industrial application value. Attached Figure Description

[0024] Figure 1 This is a diagram showing the functional modules and interaction relationships of an image acquisition device based on the Flutter platform according to the present invention. Figure 2 for Figure 1 A diagram showing the composition of each functional module in the image processing layer and their interaction with the application layer; Figure 3 This is a system architecture diagram of an image acquisition device and method based on the Flutter platform according to the present invention; Figure 4 This is a flowchart illustrating the workflow of an image acquisition method based on the Flutter platform according to the present invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments: I. Basic Terminology: 1. Flutter Engine: The core rendering engine of the Flutter platform, responsible for rendering the Dart code of Flutter applications into native image frame data to achieve cross-platform display. It is the core interface object of the image processing layer of this invention, and its native rendering logic is not modified by this invention. 2. System Display Layer: The system layer responsible for the final display of images in the terminal device. Different platforms have different underlying implementations, with CALayer for iOS and SurfaceView for Android. It is the final display carrier of the image data in this invention. 3. Frame buffer: A dedicated buffer for storing image frame data. The system display layer reads data from the frame buffer and completes the display. In this invention, the image processing layer and the system display layer frame buffer establish a standardized data stream connection to achieve synchronous data writing. 4. Callback Interface: An event-triggered data transmission mechanism. When a preset event (such as image acquisition completion) occurs, the data is automatically sent back to the caller through a predefined interface. In this invention, it is designed as a standardized input for the acquired data.

[0026] II. Innovative Terminology: 1. Image Processing Layer: This invention adds an independent lightweight layer between Flutter Engine and the system display layer. It contains only three core modules: acquisition control, data monitoring, and interface adaptation. It has no redundant functions. Its core function is to achieve accurate acquisition, transmission, and synchronization of Flutter image data without interfering with the native rendering logic of Flutter Engine. 2. Switch control module: The Boolean control logic module configured in the image processing layer of this invention contains only two states: on / off. It supports dynamic switching via Flutter application layer API. Its core function is to enable the acquisition function on demand. When off, it sleeps without additional overhead, and when on, it triggers the acquisition process. 3. Independent drawing canvas: The drawing carrier added by this invention during the collection process is generated by associating with the texture ID and is synchronized in real time with the rendering rhythm and data content of the Flutter Engine native rendering canvas. Its core function is to avoid display abnormalities caused by directly operating the display layer frame buffer and to ensure the consistency between the collected data and the displayed data. 4. Standardized Image Callback Interface: This invention provides a unified interface for image acquisition data, pre-defining a unified data format and calling specifications, supporting customized parameter adjustments. Its core function is to realize the back transmission of acquired data to the Flutter application layer and flexible integration with external extension modules, enabling a single interface to be reused across multiple platforms and modules.

[0027] like Figure 1 As shown, the present invention provides an image acquisition device based on the Flutter platform, comprising: Flutter application layer 101 is used to send start / stop commands for data collection, receive collected data, and connect to external extension modules. Flutter Engine 102 is the core rendering engine of the Flutter platform, which performs native image rendering and sends a drawing completion signal; Image processing layer 103 is a lightweight independent layer added between Flutter Engine and system display layer to realize image data acquisition, transmission, synchronization and control functions; System display layer 104 is the native display layer of the terminal device, responsible for the final display of images, and different platforms have different implementations. External extension module 105 is a functional module based on standardized callback interface, supporting functions including streaming, image analysis, recording, and real-time communication.

[0028] Figure 1 This diagram illustrates the functional modules and interaction relationships of the image acquisition device, showcasing the vertical interaction relationships and bidirectional data flow across five core levels. Each level is connected via standardized data channels, with the image processing layer serving as the core intermediary layer. This intermediary layer does not interfere with the native rendering logic of the Flutter Engine, achieving integrated data flow from acquisition to transmission, display, and expansion.

[0029] like Figure 2 As shown, the image processing layer 103 includes: The acquisition control module 1031 is responsible for the frame buffer association of the system display layer, the creation of independent synchronous canvases, and the execution of image acquisition operations; The 1032 switch control module, a Boolean control core, is used to receive API commands from the application layer and control the start and stop of the acquisition function and the signal flow between modules. The data monitoring module 1033 listens for the Flutter Engine's drawing completion signal and triggers the acquisition control module to execute the acquisition job. The interface adaptation module 1034 realizes the standardized conversion of the collected data format and completes the data return and docking with external extension modules through a unified callback interface.

[0030] Figure 2 This diagram illustrates the internal core modules and their interactions within the image processing layer, showcasing the collaborative working logic of the four core functional modules. Each module is independently encapsulated, interacts with one-way / two-way signals, achieves remote status control via application layer APIs, and implements standardized data output through interface adapter modules. The overall design is lightweight, with no redundant functional modules.

[0031] This invention provides an image acquisition device and method based on the Flutter platform, the system architecture of which is as follows: Figure 3 As shown, through five core steps—adding an image processing layer, configuring core modules, implementing state-based execution processes, and achieving cross-platform expansion—efficient, low-overhead, and scalable image acquisition effects are achieved collaboratively. The specific steps are as follows: Step 1: Add a lightweight independent image processing layer: An independent lightweight image processing layer is added between the Flutter Engine (the core rendering engine of Flutter, responsible for rendering Dart code into native image frame data) and the system display layer (the system layer responsible for the final display of images in the terminal device; CALayer for iOS and SurfaceView for Android).

[0032] This layer does not interfere with Flutter Engine's native rendering logic. It serves only as the core carrier for image data transfer, acquisition, and synchronization. It contains only three core functional modules: acquisition control module, data monitoring module, and interface adaptation module. There are no redundant functions, avoiding the introduction of additional performance burden and ensuring real-time synchronization with Flutter Engine's rendering rhythm.

[0033] Step 2: Configure the switch control module and standardized image callback interface: Two core components are configured in the image processing layer, which work together to achieve flexible control of the acquisition function and standardized data flow: 1. Switch control module: It adopts Boolean control logic and only includes two states: on / off. The core is used to control the start and stop of the image acquisition function. This module supports dynamic switching of the state through the API interface provided by the Flutter application layer to meet the on-demand activation requirements of the acquisition function in business scenarios.

[0034] 2. Standardized Image Callback Interface: A unified image data format (such as YUV420, ARGB) and interface calling specifications are predefined. The core is used to send the collected image data back to the Flutter application layer, and at the same time, it serves as a general data entry point to connect to external extension modules. This interface supports customized parameter adjustment, and the output resolution, frame rate, and data format of the image data can be flexibly adjusted according to the needs of the connected modules.

[0035] Step 3: Execute the corresponding processing flow based on the status of the switch control module: Based on the real-time status of the switch control module, a differentiated processing flow is executed to enable the data acquisition function on demand, while ensuring the normal display of the Flutter application in both states, as detailed below: 3.1 Switch off state: The image processing layer is in a dormant state and does not perform any acquisition-related operations. It follows the native Flutter rendering process: Flutter Engine directly transmits the rendered image frame data to the system display layer, which then displays the image. At this time, the image processing layer has no additional performance overhead, ensuring that the application's native performance is not affected.

[0036] 3.2 Switch on state: The image processing layer initiates and executes the acquisition process, ensuring synchronization between acquisition and display throughout the entire process, with no delay and no display anomalies. The specific sub-steps are as follows: 1. Associating the system display layer frame buffer: Establishing a standardized data stream connection between the image processing layer and the system display layer, obtaining the address, data format, and read / write specifications of the system display layer frame buffer, and ensuring the consistency of the format of the acquired data and the displayed data; 2. Added a new synchronous independent drawing canvas: An independent drawing canvas is generated by associating with the texture ID. The rendering rhythm and data content of this canvas are synchronized in real time with the native rendering canvas of Flutter Engine. This allows Flutter Engine to synchronously draw the rendered image data onto this canvas, avoiding display abnormalities caused by directly operating the system's display layer frame buffer. 3. Listen for rendering completion signals: The data listening module listens for the rendering completion signals of Flutter Engine in real time, accurately capturing the rendering completion node of each frame of the image; 4. Collect and synchronize image data: After detecting the drawing completion signal, immediately collect image data (including key information such as original pixel data, image resolution, and rendering frame rate) from the newly added independent drawing canvas. The collected data is sent back to the Flutter application layer through the standardized image callback interface. At the same time, the image data is synchronously written to the frame buffer of the system display layer to achieve normal image display. 5. Loop Acquisition: Repeat sub-steps 1 to 4 above to achieve real-time acquisition of continuous frames until the switch control module switches to the off state.

[0037] like Figure 4 As shown, the programmatic expression of the execution flow when the switch control module of the present invention is turned on is as follows: S01: Initialize acquisition resources. After calling the start API, initialize core resources such as acquisition thread, texture information, and data callback. S02: Associate the display layer frame buffer, establish a standardized data stream connection with the system display layer frame buffer, and obtain information such as frame buffer address and data format; S03: Create an independent synchronized canvas, generate an independent drawing canvas by associating it with the texture ID, and synchronize it with the Flutter Engine native rendering canvas in real time; S04: Listen for the drawing completion signal, register a drawing signal listener, and accurately capture the rendering completion node of each frame of the Flutter Engine; S05: Acquire image data, read raw image data from an independent synchronized canvas, including key information such as pixel data, resolution, and rendering frame rate; S06: Image data feedback. The collected data is fed back to the Flutter application layer through a standardized callback interface, providing a data entry point for the extended modules. S07: Synchronize to display layer display, synchronously write the acquired image data to the system display layer frame buffer, triggering normal interface refresh display; S08: Release acquisition resources. After calling the close API, all acquisition-related resources, including the canvas, listeners, and threads, are destroyed, and system memory is released.

[0038] Figure 4 This is a sequence diagram of the entire image acquisition process when the switch is turned on, including 8 core execution steps, 2 loop judgment nodes, and pre / post conditions, clearly defining the triggering conditions for the "acquisition loop" and "resource release". All steps in the process ensure "synchronous acquisition and display without delay", without direct operation of the operating system display layer frame buffer to avoid display abnormalities. After acquisition is turned off, all resources are released completely, with no memory leaks.

[0039] Step 4: Achieve cross-platform reuse and feature expansion: 1. Cross-platform reuse: Leveraging Flutter's cross-platform capabilities, the core logic of image acquisition (image processing layer, switch control module, acquisition process) is developed based on Flutter's native syntax, enabling seamless reuse of a single codebase across mobile (Android, iOS), desktop (Windows, Mac), and embedded Flutter applications, without the need for native adaptation development for different platforms; 2. Functionality Expansion: Through standardized image callback interfaces, the acquired image data can be flexibly connected to external extension modules, supporting one or more combinations of functions such as streaming, image analysis, local recording, and real-time communication. Adding new extension functions only requires developing the connection logic once, which can then be reused on all platforms.

[0040] Step 5: Resource release after the switch is turned off: When the switch control module switches from the on state to the off state, the image processing layer automatically performs resource destruction and release operations: destroying the newly added independent drawing canvas, drawing completion signal listener and related acquisition threads, releasing all system resources occupied by the image processing layer, completely restoring the pure Flutter native drawing process, and avoiding memory leak issues.

[0041] Example 1: This embodiment corresponds to a live streaming scenario on an Android mobile device. The system display layer is SurfaceView on the Android side, and the extended module is the streaming module, which realizes real-time acquisition, streaming, and normal display of the Flutter application interface. The specific implementation steps are as follows: 1. Application initialization: When a Flutter application starts, the image processing layer is initialized. By default, the switch control module is set to the off state. At this time, the image processing layer is in sleep mode. Flutter Engine transmits rendering data to SurfaceView according to the native process, and the application is displayed normally without any additional overhead. 2. Enable the capture function: After the user triggers the live streaming command, the Flutter application layer calls the setEnable(true) API of the image processing layer to switch the switch control module to the on state, and initializes the capture-related resources, including creating the image frame processing thread, the display layer data push thread, and setting the image data listening callback onRender(eglContext,textureId,width,height); 3. Associate frame buffer and create synchronous canvas: Listen to the surfaceCreated state of SurfaceView, get the drawn canvas (surface) from SurfaceView, and get the handle frameBufferId of its frame buffer through EGL; at the same time, create a new texture, associate it with the texture ID to generate an independent drawing canvas (Surface), and set onFrameAvailableListener to listen for the drawing completion signal of each frame. 4. Data Acquisition and Synchronization: After receiving the drawing completion signal, the image data (original pixel data, resolution, frame rate) is read from the texture corresponding to the independent drawing canvas. The image data is then sent back to the Flutter application layer through the onRender callback. At the same time, the image data is synchronously written to the frame buffer corresponding to frameBufferId to achieve normal screen display. 5. Extended Streaming Functionality: After receiving the returned image data, the Flutter application layer encodes it into h264 format, merges it with the acquired audio stream, and sends it to the streaming server via the RTMP protocol to achieve live streaming. 6. Disable capture and release resources: After the user ends the live stream, call the setEnable(false) API to switch the switch control module to the off state. The image processing layer automatically destroys the independent drawing canvas, onFrameAvailableListener listener and related threads, releases system resources, and restores the Flutter native drawing process.

[0042] Implementation results: The capture frame rate is stable at 60fps, the capture and display latency is less than 20ms, the system CPU usage is reduced by 40% compared with the system-level screen recording solution, and the streaming video has no screen tearing or frame skipping, fully meeting the real-time and stability requirements of live streaming.

[0043] Other application scenarios: The technical solution of this invention can also be adapted to real-time video calls in iOS Flutter applications and screen capture scenarios in desktop Flutter applications. It only needs to connect to the corresponding real-time call and screen capture modules through a standardized image callback interface. Without modifying the core acquisition logic, the function can be expanded, and the cross-platform adaptability is strong.

[0044] It should be clearly stated that the above embodiments provided by the present invention are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Equivalent modifications made based on the above embodiments are all within the scope of protection of the present invention.

Claims

1. An image acquisition device based on the Flutter platform, characterized in that, include: The Flutter application layer is used to send start / stop commands for data collection, receive collected data, and connect to external extension modules. Flutter Engine is the core rendering engine of the Flutter platform, which performs native image rendering and sends a drawing completion signal; The image processing layer is a lightweight, independent layer added between the Flutter Engine and the system display layer to realize image data acquisition, transmission, synchronization, and control functions. The system display layer, the native display layer of the terminal device, is responsible for the final display of images, and different platforms adapt to different implementations. External extension modules are functional modules that connect to standardized callback interfaces, supporting functions including streaming, image analysis, recording, and real-time communication.

2. The image acquisition device based on the Flutter platform according to claim 1, characterized in that, The image processing layer includes: The acquisition control module is responsible for associating the frame buffer of the system display layer, creating an independent synchronous canvas, and performing image acquisition operations. The switch control module, a Boolean control core, is used to receive API commands from the application layer and control the start and stop of the data acquisition function and the signal flow between modules. The data monitoring module listens for the Flutter Engine's drawing completion signal and triggers the acquisition control module to execute the acquisition job. The interface adaptation module enables standardized conversion of collected data formats and completes data return and connection with external extension modules through a unified callback interface.

3. An image acquisition method based on the Flutter platform, characterized in that, Includes the following steps: Step 1: Add a lightweight independent image processing layer between Flutter Engine and the system display layer. The image processing layer only contains acquisition control, data monitoring, and interface adaptation modules, and does not interfere with the native rendering logic of Flutter Engine, so as to realize image data acquisition, transmission and synchronization. Step 2: Configure a switch control module and a standardized image callback interface in the image processing layer. The switch control module controls the start and stop of the image acquisition function, and the image callback interface defines a unified data format and calling standard for image data return and external expansion module docking. Step 3: Execute the process according to the state of the switch control module: When closed, the native Flutter drawing process is used, and the image processing layer sleeps without additional overhead; when open, the system display layer frame buffer is associated, an independent drawing canvas synchronized with the native rendering canvas is added, and after listening for the Flutter Engine drawing completion signal, the image data in the canvas is collected. At the same time, the data is sent back through the image callback interface and synchronously written to the system display layer frame buffer to achieve normal display. Step 4: Leveraging Flutter's cross-platform capabilities, implement a single codebase for reusing the core image acquisition logic across multiple platforms. Connect to one or more extended modules from streaming, image analysis, recording, and real-time communication via the image callback interface to extend the image acquisition functionality.

4. The image acquisition method based on the Flutter platform according to claim 3, characterized in that: In step 1, the system display layer is CALayer on iOS or SurfaceView on Android. The image processing layer establishes a standardized data stream connection with the frame buffer of the display layer of different platform systems to adapt to the frame buffer data format and read / write specifications of multiple platforms.

5. The image acquisition method based on the Flutter platform according to claim 3, characterized in that, In step 2, the switch control module is a Boolean control logic module that supports dynamically switching between on and off states via the Flutter application layer API, enabling image acquisition to be enabled on demand.

6. The image acquisition method based on the Flutter platform according to claim 3, characterized in that, In step 2, when the switch control module is turned on, the collected image data includes original pixel data, resolution, and rendering frame rate information, and the collected data is consistent with the data displayed in the system display layer, with no delay in collection and display synchronization.

7. The image acquisition method based on the Flutter platform according to claim 3, characterized in that, In step 2, the independently drawn canvas is generated by associating with the texture ID and is synchronized in real time with the rendering rhythm and data content of the native rendering canvas, avoiding display abnormalities caused by directly manipulating the frame buffer.

8. The image acquisition method based on the Flutter platform according to claim 3, characterized in that, In step 2, when the switch control module is turned off, the independent drawing canvas, the drawing completion signal listener and related acquisition threads are automatically destroyed, system resources are released and the Flutter native drawing process is restored.

9. The image acquisition method based on the Flutter platform according to claim 3, characterized in that, In step 3, the process of executing the flow when the switch control module is turned on is as follows: S01: Initialize acquisition resources. After calling the start API, initialize core resources such as acquisition thread, texture information, and data callback. S02: Associate the display layer frame buffer, establish a standardized data stream connection with the system display layer frame buffer, and obtain information such as frame buffer address and data format; S03: Create an independent synchronized canvas, generate an independent drawing canvas by associating it with the texture ID, and synchronize it with the Flutter Engine native rendering canvas in real time; S04: Listen for the drawing completion signal, register a drawing signal listener, and accurately capture the rendering completion node of each frame of the Flutter Engine; S05: Acquire image data, read raw image data from an independent synchronized canvas, including key information such as pixel data, resolution, and rendering frame rate; S06: Image data feedback. The collected data is fed back to the Flutter application layer through a standardized callback interface, providing a data entry point for the extended modules. S07: Synchronize to display layer display, synchronously write the acquired image data to the system display layer frame buffer, triggering normal interface refresh display; S08: Release acquisition resources. After calling the close API, all acquisition-related resources, including the canvas, listeners, and threads, are destroyed, and system memory is released.

Images