A multi-source picture real-time synthesis and virtual director output method based on an Android system

Abstract

This invention discloses a method for real-time compositing and virtual broadcast output of multi-source images based on the Android system, comprising: differentially acquiring and preprocessing heterogeneous input sources to obtain input source images in a unified format; the user configuring the layout mode of the composite image through an interactive page, generating and storing composite image frame data in real time based on the images from each input source; registering a virtual broadcast device at the HAL layer, which is configured to be presented as a logical camera device to the Android camera service; distributing the data to the upper-layer application calling the virtual broadcast device through the Android camera service; and other Android applications obtaining the composite image frame data through the standard Camera API. This invention solves the problems of existing technologies, such as composite images being limited to use within the application, inability to achieve system-level virtual camera output, insufficient compatibility with heterogeneous input sources, and lack of a virtual broadcast scheduling mechanism.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology for Android smart terminals, and specifically relates to a method for real-time synthesis of multi-source images and virtual broadcast output based on the Android system. Background Technology

[0002] With the widespread adoption of mobile smart terminals and the rapid development and extensive application of scenarios such as live video streaming, video conferencing, online education, and security monitoring, the demand for real-time compositing of multi-source images is becoming increasingly prominent. Traditional Android terminals typically output images from a single data source (native camera, local video, network stream, etc.), which cannot meet the needs of scenarios that require merging multiple heterogeneous data sources such as camera previews, PDF documents, local recordings, and RTSP network video streams into a single composite image.

[0003] In existing technologies, multi-screen compositing within Android applications typically employs OpenGL texture blending or SurfaceFlinger layer compositing, rendering and compositing multiple input source images at the application layer before directly displaying them on the current application interface. These solutions share a common problem: the composited image can only be displayed and used within the current application and cannot be output as a system-level "virtual camera" for use by other third-party applications. When users wish to use the composited image for scenarios such as video conferencing and live streaming, existing solutions cannot meet this requirement.

[0004] For example, Chinese patent CN112767231B discloses a layer compositing method and device. This invention does not provide a system-level virtual camera device node, which prevents third-party applications from accessing the composited image; furthermore, the layer source of this invention is not compatible with multiple heterogeneous sources and does not support the compositing of multiple types of media. Summary of the Invention

[0005] The purpose of this invention is to provide a method for real-time compositing of multi-source images and virtual broadcast output based on the Android system, so as to solve the problems of existing multi-image compositing schemes in Android applications, which have the problem that the synthesized images can only be used within the application, cannot realize system-level virtual camera output, and have insufficient compatibility with heterogeneous input sources and lack a virtual broadcast scheduling mechanism.

[0006] The technical solution of the present invention is as follows: On one hand, this invention provides a method for real-time synthesis of multi-source images and virtual broadcast output based on the Android system, including the following steps: The virtual director app performs differentiated acquisition and preprocessing of heterogeneous input sources to obtain input source images in a unified format; Users configure the layout mode of the composite image through the interactive page, and the composite image frame data is generated in real time based on the images from each input source. A new HIDL service module is added to the Android system for storing and forwarding composite image frame data; a virtual director device is registered in the HAL layer, which is configured to be presented as a standard logical camera device to the Android camera service; The compositing engine writes each frame of video data into a shared memory area managed by the HIDL service module; the Android camera service reads the composited video frame data from the shared memory area, injects the video frame data into the camera hardware buffer of the Android system according to the standard camera data reporting process, and distributes it to other Android applications that are calling the virtual broadcasting device. Other Android applications obtain composite image frame data through the standard Camera API.

[0007] Preferably, the differentiated acquisition and preprocessing of heterogeneous input sources specifically includes: For camera preview streams, real-time YUV image frame data is obtained via the Camera API; For PDF documents, the PDF rendering engine converts the document pages into a sequence of Bitmap images; For local video recording files, the MediaExtractor and MediaCodec are used for decoding, and the decoding parameters are configured to extract video frame data. For RTSP network streams, a connection is established through the RTSP client library, and MediaCodec is used for hardware decoding to obtain video frame data.

[0008] Preferably, the step of generating composite image frame data in real time based on the images from each input source specifically includes: The compositing engine uses a graphics processing pipeline to load the various input source images as textures onto the GPU. The fragment shader performs texture sampling, blending, and transformation operations to generate the final composite image frame data.

[0009] Preferably, the other Android applications obtain the composite image frame data through the standard Camera API, specifically: Minimize the virtual director app and set parameters in the HIDL service module. When other Android applications enumerate the list of camera devices in the Android system through the standard Camera API, they can obtain the logical camera device and its supported parameter configuration. When other Android applications open the camera normally, the Camera HAL layer reads the parameters of the HIDL service module to determine whether to open the virtual director device. If it is determined that the virtual broadcasting device needs to be opened, the composite frame data in the shared memory area managed by the HIDL service module is read, and then the composite frame data is injected into the camera hardware buffer of the Android system according to the standard camera data reporting process. Other Android applications can obtain the composite frame data by opening the camera.

[0010] On the other hand, the present invention provides a multi-source image real-time synthesis and virtual broadcast output system based on the Android system, including an input source acquisition and preprocessing module, a multi-source image layout configuration and real-time synthesis module, a HIDL service module and a virtual broadcast device establishment module, a HIDL transmission and distribution module for synthesized image frames, and an upper-layer application calling module. The input source acquisition and preprocessing module is used by the virtual director APP to perform differentiated acquisition and preprocessing of heterogeneous input sources to obtain input source images in a unified format; The multi-source image layout configuration and real-time compositing module is used by users to configure the layout mode of the composite image through an interactive page, and to generate composite image frame data in real time based on the input source images. The HIDL service module and the virtual broadcasting device establishment module are used to add a new HIDL service module in the Android system for storing and forwarding composite image frame data; and to register a virtual broadcasting device in the HAL layer, wherein the virtual broadcasting device is configured to be presented as a standard logical camera device to the Android camera service. The HIDL transmission and distribution module for composited video frames is used by the compositing engine to write each frame of video frame data into a shared memory area managed by the HIDL service module; the Android camera service reads the composited video frame data from the shared memory area, injects the video frame data into the camera hardware buffer of the Android system according to the standard camera data reporting process, and distributes it to other Android applications that are calling the virtual broadcasting device. The upper-layer application calls this module, which is used by other Android applications to obtain composite image frame data through the standard Camera API.

[0011] Preferably, the input source acquisition and preprocessing module performs differentiated acquisition and preprocessing on heterogeneous input sources, specifically as follows: For camera preview streams, real-time YUV image frame data is obtained via the Camera API; For PDF documents, the PDF rendering engine converts the document pages into a sequence of Bitmap images; For local video recording files, the MediaExtractor and MediaCodec are used for decoding, and the decoding parameters are configured to extract video frame data. For RTSP network streams, a connection is established through the RTSP client library, and MediaCodec is used for hardware decoding to obtain video frame data.

[0012] Preferably, the layout configuration and real-time compositing module for the multi-source images generates composite image frame data in real time based on each input source image, specifically as follows: The compositing engine uses a graphics processing pipeline to load the various input source images as textures onto the GPU. The fragment shader performs texture sampling, blending, and transformation operations to generate the final composite image frame data.

[0013] Preferably, other Android applications in the calling module of the upper-layer application obtain the composite image frame data through the standard Camera API, specifically: Minimize the virtual director app and set parameters in the HIDL service module. When other Android applications enumerate the list of camera devices in the Android system through the standard Camera API, they can obtain the logical camera device and its supported parameter configuration. When other Android applications open the camera normally, the Camera HAL layer reads the parameters of the HIDL service module to determine whether to open the virtual director device. If it is determined that the virtual broadcasting device needs to be opened, the composite frame data in the shared memory area managed by the HIDL service module is read, and then the composite frame data is injected into the camera hardware buffer of the Android system according to the standard camera data reporting process. Other Android applications can obtain the composite frame data by opening the camera.

[0014] In another aspect, the present invention also provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any embodiment of the present invention.

[0015] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0016] Compared with the prior art, the present invention has the following technical effects: 1. Integration capability of multi-source heterogeneous images: This invention realizes the unified acquisition and fusion of inputs of different natures, such as real-time camera images, static PDF documents, local videos, and RTSP network streams, and presents them in the same screen. This breaks the limitation of single data source output of traditional Android terminals and adapts to the input needs of multiple scenarios.

[0017] 2. System-level virtualization output of composited images: The composited broadcast images of this invention are no longer limited to the display within the current application. The virtual broadcast module of the HAL layer is registered as a system virtual camera service and can be called by any third-party application as a standard camera, realizing system-level reuse of composited images without the need for each application to develop its own composite function.

[0018] 3. Achieve stable cross-layer transmission: This invention utilizes the Android standard HIDL framework to complete cross-process data transmission between the application layer and the HAL layer. Through standardized data structure interfaces, it avoids the overhead of multiple copies during data transmission, ensuring the stability, real-time smoothness, and compatibility of the composite screen transmission, while achieving low-power operation and adapting to different Android system versions.

[0019] 4. Enhanced scene adaptability: The system described in this invention adopts a modular design (input source module, synthesis engine, and transmission module are decoupled from each other), which can be widely used in live streaming, video conferencing, in-vehicle multimedia, security monitoring, educational screen projection and other fields. It solves the problems of the inability to call synthesized images at the system level and the difficulty of multi-source fusion in the existing technology, and has strong practicality. Attached Figure Description

[0020] Figure 1 This is an overall flowchart of the multi-source real-time image synthesis and virtual broadcast output method based on the Android system described in this invention; Figure 2 This is a flowchart of differentiated acquisition and preprocessing for multiple heterogeneous input sources; Figure 3 This is a flowchart of a camera application acquiring composite image frame data from a virtual broadcasting device. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0022] Example 1 The purpose of this invention is to solve three core problems: multi-source fusion, system-level output, and unified access to virtual cameras, and to achieve the following technical objectives: It supports unified acquisition, decoding, and image extraction from various heterogeneous input sources, including camera preview streams, PDF documents, local video files, and RTSP network video streams.

[0023] Real-time compositing of multiple input source images is implemented at the Android application layer, supporting various compositing layouts such as picture-in-picture, split-screen, and overlay.

[0024] The composite image is transmitted to the hardware abstraction layer through the standard HIDL framework to establish a virtual broadcasting device node.

[0025] This allows other Android applications to access the virtual broadcasting device just like they would access a regular camera device, and to acquire and use the composited image data.

[0026] Therefore, this embodiment provides a method for real-time compositing of multi-source images and virtual broadcast output based on the Android system. It is implemented entirely based on the Android standard framework (Camera API, MediaCodec, OpenGL ES, HIDL HAL), and can run on various devices running Android 8.0 and above, exhibiting good compatibility and portability. (See also...) Figure 1 As shown, it includes the following steps: The virtual director app performs differentiated acquisition and preprocessing of heterogeneous input sources to obtain input source images in a unified format. Specifically, a multi-source input management module is established at the Android application layer. This module supports access from at least four types of input sources: camera data sources, PDF data sources, local video recording data sources, and RTSP network data sources. Users can select one or more of these input sources to combine according to their actual needs.

[0027] As a preferred embodiment of this practice, the differentiated acquisition and preprocessing of heterogeneous input sources, such as... Figure 2 As shown, specifically: For camera preview streams, real-time YUV image frame data is obtained through Camera APIs such as the Android Camera2 API or CameraX API.

[0028] For PDF documents, the PDF rendering engine converts the document pages into a sequence of bitmap images.

[0029] For local video files, MediaExtractor and MediaCodec are used to decode common video files such as MP4, AVI, and MKV, and the decoding parameters are configured to extract video frame data.

[0030] For RTSP network streams, a connection is established through the RTSP client library, and MediaCodec is used for hardware decoding to obtain video frame data.

[0031] All acquired data frames are uniformly converted into image buffers in YUV420 or RGB format.

[0032] This innovative approach integrates four different types of input sources—local camera footage, PDF documents, local video, and RTSP network streams—into a unified acquisition and preprocessing framework, solving the problem of unifying and integrating multiple input sources in existing technologies. Whether it's real-time captured footage, electronic document content, local materials, or remote video streams, all can be used as components of the composite image, supporting switching and meeting the needs of multi-scene, multi-view fusion.

[0033] Users configure the layout mode of the composite image through an interactive page, generating composite image frame data in real time based on the images from each input source. Specifically, this embodiment supports multiple image layout modes, allowing users to freely adjust the position, size, layer, and visual effects of each input source image as needed. Layout modes may include picture-in-picture mode (one main image plus one or more smaller images), split-screen mode (such as dual-screen two-part screen, four-screen four-part screen), and so on.

[0034] As a preferred embodiment of this practice, the step of generating composite image frame data in real time based on the images from each input source specifically involves: The compositing engine employs a graphics processing pipeline (such as OpenGL ES) to load the images from each input source as textures onto the GPU. The fragment shader then performs texture sampling, blending, and transformation operations to generate the final composite image frame data. The compositing process supports real-time frame rate synchronization, ensuring that the images from each input source are aligned on the timeline.

[0035] A new HIDL service module is added to the Android system. This HIDL service module is responsible for managing the shared memory area used to store composite image frame data. A virtual director device (VirtualCameraHAL) is registered in the HAL layer. This virtual director device implements the standard ICameraProvider and ICameraDevice HIDL interfaces and is configured to present itself as a standard logical camera device to the Android camera service. The virtual director device and the virtual director APP are correspondingly linked, with the virtual director APP handling multi-channel input and compositing.

[0036] This step registers the composited image from the application layer as a system-level virtual camera device using the HIDL framework, enabling cross-process communication between the application layer and the HAL layer. This provides strong compatibility and high stability, allowing the composited image to be invoked by any third-party application. This design overcomes the limitation of existing technologies where composited images can only be embedded within the current application, expanding application scenarios.

[0037] The compositing engine writes each frame of video data into a shared memory area managed by the HIDL service module; the Android camera service reads the composited video frame data from the shared memory area, injects the video frame data into the camera hardware buffer of the Android system according to the standard camera data reporting process, and distributes it to other Android applications that are calling the virtual broadcasting device.

[0038] This step constructs a virtual broadcasting mechanism for the HAL layer through shared memory managed by the HIDL service module, enabling cross-process data transmission. This allows all camera applications within the system to open the composited image by default, avoiding the performance overhead caused by data copying, ensuring the real-time performance and smoothness of the composited image, and providing system-level reuse capabilities.

[0039] Other Android applications obtain composite frame data through standard Camera APIs (such as Camera1 API or Camera2 API).

[0040] In a preferred embodiment of this practice, other Android applications in the calling module of the upper-layer application obtain composite frame data through the standard Camera API, such as... Figure 3 The diagram illustrates the interactive process of a camera application reading footage from a virtual broadcasting device when the camera is activated. Specifically: Minimize the virtual director app (which can be minimized to a floating ball) and set parameters (such as key) in the HIDL service module. When other Android applications enumerate the list of camera devices in the Android system through the standard Camera API, they can obtain the logical camera device and its supported parameter configuration. When other Android applications open the camera normally, the Camera HAL layer reads the parameters of the HIDL service module to determine whether to open the virtual director device (by checking if the key has a value).

[0041] If it is determined that the virtual broadcasting device needs to be opened, the composite frame data in the shared memory area managed by the HIDL service module is read, and then the composite frame data is injected into the camera hardware buffer of the Android system according to the standard camera data reporting process. Other Android applications can obtain the composite frame data by opening the camera.

[0042] Example 2 Accordingly, this embodiment provides a multi-source video real-time synthesis and virtual broadcast output system based on the Android system, which is used to implement the method described in any embodiment of the present invention, including an input source acquisition and preprocessing module, a multi-source video layout configuration and real-time synthesis module, a HIDL service module and a virtual broadcast device establishment module, a HIDL transmission and distribution module for synthesized video frames, and a calling module for upper-layer applications. The input source acquisition and preprocessing module is used by the virtual director APP to perform differentiated acquisition and preprocessing of heterogeneous input sources to obtain input source images in a unified format; The multi-source image layout configuration and real-time compositing module is used by users to configure the layout mode of the composite image through an interactive page, and to generate composite image frame data in real time based on the input source images. The HIDL service module and the virtual broadcasting device establishment module are used to add a new HIDL service module in the Android system for storing and forwarding composite image frame data; and to register a virtual broadcasting device in the HAL layer, wherein the virtual broadcasting device is configured to be presented as a standard logical camera device to the Android camera service. The HIDL transmission and distribution module for composited video frames is used by the compositing engine to write each frame of video frame data into a shared memory area managed by the HIDL service module; the Android camera service reads the composited video frame data from the shared memory area, injects the video frame data into the camera hardware buffer of the Android system according to the standard camera data reporting process, and distributes it to other Android applications that are calling the virtual broadcasting device. The upper-layer application calls this module, which is used by other Android applications to obtain composite image frame data through the standard Camera API.

[0043] In a preferred embodiment of this invention, the input source acquisition and preprocessing module performs differentiated acquisition and preprocessing on heterogeneous input sources, specifically as follows: For camera preview streams, real-time YUV image frame data is obtained via the Camera API; For PDF documents, the PDF rendering engine converts the document pages into a sequence of Bitmap images; For local video recording files, the MediaExtractor and MediaCodec are used for decoding, and the decoding parameters are configured to extract video frame data. For RTSP network streams, a connection is established through the RTSP client library, and MediaCodec is used for hardware decoding to obtain video frame data.

[0044] In a preferred embodiment of this invention, the multi-source image layout configuration and real-time compositing module generates composite image frame data in real time based on each input source image, specifically as follows: The compositing engine uses a graphics processing pipeline to load the various input source images as textures onto the GPU. The fragment shader performs texture sampling, blending, and transformation operations to generate the final composite image frame data.

[0045] In a preferred embodiment of this practice, other Android applications in the calling module of the upper-layer application obtain the composite image frame data through the standard Camera API, specifically as follows: Minimize the virtual director app and set parameters in the HIDL service module. When other Android applications enumerate the list of camera devices in the Android system through the standard Camera API, they can obtain the logical camera device and its supported parameter configuration. When other Android applications open the camera normally, the Camera HAL layer reads the parameters of the HIDL service module to determine whether to open the virtual director device. If it is determined that the virtual broadcasting device needs to be opened, the composite frame data in the shared memory area managed by the HIDL service module is read, and then the composite frame data is injected into the camera hardware buffer of the Android system according to the standard camera data reporting process. Other Android applications can obtain the composite frame data by opening the camera.

[0046] Example 3 This embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in any embodiment of the present invention.

[0047] Example 4 This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0048] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0049] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0050] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0051] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0052] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for real-time synthesis and virtual broadcast output of multi-source video based on the Android system, characterized in that, Includes the following steps: The virtual director app performs differentiated acquisition and preprocessing of heterogeneous input sources to obtain input source images in a unified format; Users configure the layout mode of the composite image through the interactive page, and the composite image frame data is generated in real time based on the images from each input source. A new HIDL service module is added to the Android system for storing and forwarding composite image frame data; a virtual director device is registered in the HAL layer, which is configured to be presented as a standard logical camera device to the Android camera service; The compositing engine writes each frame of video data into a shared memory area managed by the HIDL service module; the Android camera service reads the composited video frame data from the shared memory area, injects the video frame data into the camera hardware buffer of the Android system according to the standard camera data reporting process, and distributes it to other Android applications that are calling the virtual broadcasting device. Other Android applications obtain composite image frame data through the standard Camera API.

2. The method for real-time synthesis and virtual broadcast output of multi-source images based on the Android system according to claim 1, characterized in that, The differentiated acquisition and preprocessing of heterogeneous input sources specifically includes: For camera preview streams, real-time YUV image frame data is obtained via the Camera API; For PDF documents, the PDF rendering engine converts the document pages into a sequence of Bitmap images; For local video recording files, the MediaExtractor and MediaCodec are used for decoding, and the decoding parameters are configured to extract video frame data. For RTSP network streams, a connection is established through the RTSP client library, and MediaCodec is used for hardware decoding to obtain video frame data.

3. The method for real-time synthesis and virtual broadcast output of multi-source images based on the Android system according to claim 1, characterized in that, The process of generating composite image frame data in real time based on the images from each input source is as follows: The compositing engine uses a graphics processing pipeline to load the various input source images as textures onto the GPU. The fragment shader performs texture sampling, blending, and transformation operations to generate the final composite image frame data.

4. The method for real-time synthesis and virtual broadcast output of multi-source images based on the Android system according to claim 1, characterized in that, The other Android applications obtain composite image frame data through the standard Camera API, specifically: Minimize the virtual director app and set parameters in the HIDL service module. When other Android applications enumerate the list of camera devices in the Android system through the standard Camera API, they obtain the logical camera device and its supported parameter configuration. When other Android applications open the camera normally, the Camera HAL layer reads the parameters of the HIDL service module to determine whether to open the virtual director device. If it is determined that the virtual broadcasting device needs to be opened, the composite frame data in the shared memory area managed by the HIDL service module is read, and then the composite frame data is injected into the camera hardware buffer of the Android system according to the standard camera data reporting process. Other Android applications can obtain the composite frame data by opening the camera.

5. A multi-source video real-time compositing and virtual broadcast output system based on the Android system, characterized in that, It includes an input source acquisition and preprocessing module, a multi-source image layout configuration and real-time synthesis module, a HIDL service module and a virtual broadcasting equipment establishment module, a HIDL transmission and distribution module for synthesized image frames, and an upper-layer application calling module. The input source acquisition and preprocessing module is used by the virtual director APP to perform differentiated acquisition and preprocessing of heterogeneous input sources to obtain input source images in a unified format; The multi-source image layout configuration and real-time compositing module is used by users to configure the layout mode of the composite image through an interactive page, and to generate composite image frame data in real time based on the input source images. The HIDL service module and the virtual broadcasting device establishment module are used to add a new HIDL service module in the Android system for storing and forwarding composite image frame data; and to register a virtual broadcasting device in the HAL layer, wherein the virtual broadcasting device is configured to be presented as a standard logical camera device to the Android camera service. The HIDL transmission and distribution module for composited video frames is used by the compositing engine to write each frame of video frame data into a shared memory area managed by the HIDL service module; the Android camera service reads the composited video frame data from the shared memory area, injects the video frame data into the camera hardware buffer of the Android system according to the standard camera data reporting process, and distributes it to other Android applications that are calling the virtual broadcasting device. The upper-layer application calls this module, which is used by other Android applications to obtain composite image frame data through the standard Camera API.

6. The multi-source real-time image synthesis and virtual broadcast output system based on the Android system according to claim 5, characterized in that, The input source acquisition and preprocessing module performs differentiated acquisition and preprocessing on heterogeneous input sources, specifically as follows: For camera preview streams, real-time YUV image frame data is obtained via the Camera API; For PDF documents, the PDF rendering engine converts the document pages into a sequence of Bitmap images; For local video recording files, the MediaExtractor and MediaCodec are used for decoding, and the decoding parameters are configured to extract video frame data. For RTSP network streams, a connection is established through the RTSP client library, and MediaCodec is used for hardware decoding to obtain video frame data.

7. The multi-source real-time image synthesis and virtual broadcast output system based on the Android system according to claim 5, characterized in that, The multi-source image layout configuration and real-time compositing module generates composite image frame data in real time based on each input source image, specifically as follows: The compositing engine uses a graphics processing pipeline to load the various input source images as textures onto the GPU. The fragment shader performs texture sampling, blending, and transformation operations to generate the final composite image frame data.

8. The multi-source real-time image synthesis and virtual broadcast output system based on the Android system according to claim 5, characterized in that, In the calling module of the upper-layer application, other Android applications obtain composite image frame data through the standard Camera API, specifically as follows: Minimize the virtual director app and set parameters in the HIDL service module. When other Android applications enumerate the list of camera devices in the Android system through the standard Camera API, they obtain the logical camera device and its supported parameter configuration. When other Android applications open the camera normally, the Camera HAL layer reads the parameters of the HIDL service module to determine whether to open the virtual director device. If it is determined that the virtual broadcasting device needs to be opened, the composite frame data in the shared memory area managed by the HIDL service module is read, and then the composite frame data is injected into the camera hardware buffer of the Android system according to the standard camera data reporting process. Other Android applications can obtain the composite frame data by opening the camera.

9. An electronic device, the electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images