Implementation Method, Device, Equipment and Storage Medium of Multimedia Data Transmission System

Abstract

A method, apparatus, device, and storage medium for implementing a multimedia data transmission system, which relates to the field of computers. The method includes parsing a target XML configuration file to obtain information about the link layer to be run, information about the modules to be connected, and information about the device parameters to be filled; constructing a link layer object containing a link structure based on the information about the link layer to be run and the information about the modules to be connected; constructing multiple module layer objects containing module structures through the information about the modules to be connected, connecting them to the link layer object, and creating a main thread task for the module layer objects; constructing a device layer object containing a device structure corresponding to the module layer object, connecting it to the module layer object, and filling the device structure based on the information about the device parameters to be filled; the link layer object controls the main thread task to call a target driver based on a device callback function through the link structure, and realizes data transmission through the target driver, the module structure, and the device structure, so as to achieve flexible integration of the IP core and reduce the development cycle.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for implementing a multimedia data transmission system. Background Art

[0002] In the field of image processing, the raw image data collected by a sensor usually needs to be processed by multiple IP cores (Intellectual Property Cores) on a SoC (System on Chip) before being finally transmitted to a terminal device for display or storage; among them, an IP core refers to a reusable and verified logic function module or design unit in integrated circuit design, which includes but is not limited to an image signal processor (ISP), an image enhancement module, an encoder, etc., and they together constitute a complete image processing link. When using IP cores, sometimes a single IP core is needed, and sometimes multiple IP cores need to be combined for use. Therefore, a software framework with high flexibility, short development cycle and easy use is required.

[0003] In related technologies, GStreamer, as a widely used open-source multimedia framework, is usually used to build multimedia pipelines for audio, video and image processing. However, GStreamer has some limitations and deficiencies in actual applications and specific scenarios. For example, when implementing new IP core integration, it is necessary to modify the existing upper-layer framework code, resulting in a long development cycle; and after writing the calls of multiple IP cores into an application program, if multiple IP cores need to be combined for use, it is necessary to modify the application code again and compile and run it, which further leads to a long development cycle. It can be seen that how to efficiently implement the flexible integration of IP cores to reduce the development cycle of the multimedia data transmission system is an urgent problem to be solved currently. Summary of the Invention

[0004] This application provides a method, device, equipment and storage medium for implementing a multimedia data transmission system, which can efficiently implement the flexible integration of IP cores to effectively reduce the development cycle of the multimedia data transmission system.

[0005] In a first aspect, an embodiment of this application provides a method for implementing a multimedia data transmission system, including the following steps:

[0006] Parse a preset target XML configuration file to obtain information of a layer to be run, information of modules to be connected, and information of device parameters to be filled;

[0007] Construct a link layer object containing a link structure based on the link layer information to be run and the module information to be connected. The link structure is used to store the information of the head and tail module layer objects and the operation interface functions.

[0008] Construct multiple module layer objects containing module structures through the module information to be connected and connect them to the link layer object, and create a main thread task for the module layer object. The module structure is used to store the binding information, and the binding information includes the binding relationships between the module layer object, the link layer object, the device layer object, and the device callback function.

[0009] Construct a device layer object containing a device structure corresponding to the module layer object and connect it to the corresponding module layer object, and fill the device structure based on the device parameter information to be filled.

[0010] Among them, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, the module structure, and the device structure.

[0011] Combined with the first aspect, in one implementation, the operation interface functions include a link layer initialization function and an activation function, and the device callback functions include a device initialization function and a data processing function.

[0012] Combined with the first aspect, in one implementation, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, including:

[0013] The link layer object controls each module layer object to be in an initialization state through the link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0014] Combined with the first aspect, in one implementation, the method further includes:

[0015] Create a buffer queue for the module layer object according to the preset module type corresponding to the module layer object. The buffer queue is used to realize data flow between different module layer objects.

[0016] Among them, the module structure is also used to store buffer queue information and front and back level module information.

[0017] Combined with the first aspect, in one implementation, the realization of data transmission through the target driver, the module structure, and the device structure includes:

[0018] The link layer object controls each module layer object to be in an active state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read out the target buffer from the buffer queue and submit it to the target driver;

[0019] The target driver operates on the target hardware device corresponding to the device structure based on the target buffer;

[0020] After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the front and back module information to implement data transmission.

[0021] Combined with the first aspect, in an embodiment, the reading out the target buffer from the buffer queue and submitting it to the target driver includes:

[0022] The module layer object reads out the target buffer from the buffer queue and determines whether the target buffer is the buffer that itself needs to process according to the target mark in the target buffer;

[0023] If so, submit the target buffer to the target driver;

[0024] If not, transfer the target buffer to the corresponding target buffer queue in the next-level module layer object for the next-level module layer object to execute the step of reading out the target buffer from the buffer queue based on the target buffer queue.

[0025] Combined with the first aspect, in an embodiment, the module type includes an output type, an input type, and an input / output type. The creating the buffer queue corresponding to the module layer object according to the preset module type includes:

[0026] When the module type is the output type, create an output idle queue and an output busy queue as the buffer queue of the module layer object;

[0027] When the module type is the input type, create an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object;

[0028] When the module type is the input / output type, create an output idle queue, an output busy queue, an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

[0029] In a second aspect, an embodiment of the present application provides a multimedia data transmission system implementation device, including:

[0030] An information parsing unit, which is used to parse a preset target XML configuration file to obtain information about the link layer to be run, information about the modules to be connected, and information about the device parameters to be filled;

[0031] A first creation unit, which is used to construct a link layer object containing a link structure based on the information about the link layer to be run and the information about the modules to be connected. The link structure is used to store information about the head and tail module layer objects and operation interface functions;

[0032] A second creation unit, which is used to construct multiple module layer objects containing module structures through the information about the modules to be connected and connect them to the link layer object, and create a main thread task for the module layer object. The module structure is used to store binding information, and the binding information includes the binding relationships between the module layer object and the link layer object, the device layer object, and the device callback function;

[0033] A third creation unit, which is used to construct a device layer object containing a device structure corresponding to the module layer object and connect it to the corresponding module layer object, and fill the device structure based on the information about the device parameters to be filled;

[0034] Among them, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, the module structure, and the device structure.

[0035] Combined with the second aspect, in an implementation manner, the operation interface functions include a link layer initialization function and an activation function, and the device callback functions include a device initialization function and a data processing function.

[0036] Combined with the second aspect, in an implementation manner, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, including:

[0037] The link layer object controls each module layer object to be in an initialization state through the link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0038] Combined with the second aspect, in an implementation manner, the second creation unit is further used for:

[0039] Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object. The buffer queue is used to realize data flow transfer between different module layer objects;

[0040] Among them, the module structure is further used to store buffer queue information and information about the previous and next level modules.

[0041] In combination with the second aspect, in one embodiment, the data transmission implemented through target-driven, module structure body, and device structure body includes:

[0042] The link layer object controls each module layer object to be in an active state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver;

[0043] The target driver operates on the target hardware device corresponding to the device structure body based on the target buffer;

[0044] After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the front and back module information to achieve data transmission.

[0045] In combination with the second aspect, in one embodiment, the step of reading the target buffer from the buffer queue and submitting it to the target driver includes:

[0046] The module layer object reads the target buffer from the buffer queue and determines whether the target buffer is the buffer that itself needs to process according to the target mark in the target buffer;

[0047] If so, submit the target buffer to the target driver;

[0048] If not, transfer the target buffer to the corresponding target buffer queue in the next-level module layer object for the next-level module layer object to execute the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0049] In combination with the second aspect, in one embodiment, the module types include output type, input type, and input-output type. The creation of the buffer queue corresponding to the module layer object according to the preset module type includes:

[0050] When the module type is the output type, create an output idle queue and an output busy queue as the buffer queue of the module layer object;

[0051] When the module type is the input type, create an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object;

[0052] When the module type is the input-output type, create an output idle queue, an output busy queue, an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

[0053] In a third aspect, an embodiment of the present application provides a device for implementing a multimedia data transmission system. The device for implementing the multimedia data transmission system includes a processor, a memory, and a multimedia data transmission system implementation program stored on the memory and executable by the processor. When the multimedia data transmission system implementation program is executed by the processor, the steps of the multimedia data transmission system implementation method as described above are implemented.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium with a multimedia data transmission system implementation program stored thereon. When the multimedia data transmission system implementation program is executed by a processor, the steps of the multimedia data transmission system implementation method as described above are implemented.

[0055] The beneficial effects brought by the technical solution provided by the embodiment of the present application include:

[0056] By parsing the target XML configuration file to obtain the to-be-run link layer information, to-be-connected module information, and to-be-filled device parameter information, and constructing a link layer object containing a link structure based on the to-be-run link layer information and the to-be-connected module information; then constructing multiple module layer objects containing module structures based on the to-be-connected module information and connecting them to the link layer object, and creating a main thread task for the module layer object, and then constructing a device layer object containing a device structure corresponding to the module layer object and connecting it to the corresponding module layer object to form a hierarchical architecture, and filling the device structure based on the to-be-filled device parameter information; wherein, the link layer object controls the main thread task in the module layer object to call the target driver based on the device callback function through the link structure, and realizes data transmission through the target driver, the module structure, and the device structure. It can be seen that the present application runs the new hardware device or multiple combined hardware devices that are expected to be used in the form of a configuration file to efficiently realize the flexible integration of IP cores without recompiling the application program, and realizes the flexible control of hardware operations through the callback function mechanism without modifying the upper-layer framework code, effectively shortening the development cycle of the multimedia data transmission system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic flowchart of an embodiment of the multimedia data transmission system implementation method of the present application;

[0058] Figure 2 It is a schematic diagram of the framework of the multimedia data transmission system involved in the embodiment solution of the present application;

[0059] Figure 3 For the present application Figure 1 It is a schematic flowchart of the refinement of step S40 in the present application;

[0060] Figure 4Schematic diagram of the functional modules of the implementation device of the multimedia data transmission system of this application;

[0061] Figure 5 Schematic diagram of the hardware structure of the implementation device of the multimedia data transmission system involved in the solution of the embodiment of this application. Detailed implementation manners

[0062] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0063] First, some technical terms in this application are explained to facilitate the understanding of this application by those skilled in the art.

[0064] API (Application Programming Interface): Application Programming Interface.

[0065] GDC (Geometric Distortion Correctio): Geometric Distortion Correction.

[0066] TEE (T-distributed Estimator of Error): It is the tee command used in the Linux system, which is used to represent a T-shaped structure, similar to the shunt function of a pipeline, that is, the input is sent to multiple output positions simultaneously.

[0067] To make the purpose, technical solutions, and advantages of this application clearer, the following will further describe the implementation manners of this application in detail with reference to the accompanying drawings.

[0068] In a first aspect, an embodiment of this application provides a method for implementing a multimedia data transmission system.

[0069] In one embodiment, refer to Figure 1 , Figure 1 which is the flowchart of the embodiment of the method for implementing the multimedia data transmission system of this application. As Figure 1 shown, the method for implementing the multimedia data transmission system includes:

[0070] Step S10: Parse a preset target XML configuration file to obtain the to-be-run link layer information, the to-be-connected module information, and the to-be-filled device parameter information.

[0071] Exemplary, it should be understood that currently, the construction of a multimedia data transmission system is often implemented through GStreamer. However, the architecture and API design of GStreamer are relatively complex, and the entry threshold is relatively high. That is, developers need to master a large number of details about plugins, pipelines, elements, and data stream management in order to effectively develop and optimize applications. Moreover, when integrating new IP cores, beginners must deeply understand the internal architecture of GStreamer and adapt under its API limitations, resulting in a relatively long development and debugging cycle. It should be noted that the IP core in this embodiment refers to a real hardware device.

[0072] See Figure 2 As shown, in this embodiment, in order to effectively reduce the development difficulty and development cycle, a simple hierarchical architecture consisting of a link layer Pipeline, a module layer Module, and a device layer Device will be constructed to provide a flexible and extensible data processing architecture, aiming to efficiently manage the data flow between multiple IP cores or modules in the SoC, thereby ensuring the efficient transmission and processing of data between different modules. Among them, the link layer Pipeline is mainly responsible for the connection and scheduling between module layers to achieve the orderly flow of data streams through a unified Pipeline structure; the module layer Module is mainly responsible for abstracting the hardware devices, that is, each module layer encapsulates the specific functional logic of its corresponding hardware device, etc., to process and transfer data. However, the actual hardware operations are completed by the device driver device_driver; the device layer Device is directly associated with the specific hardware device to be responsible for calling the driver to complete the actual transmission, processing, and hardware operations of the data.

[0073] In addition, in this embodiment, the new hardware device or multiple combined hardware devices that are expected to be used will be run in the form of a configuration file to efficiently achieve the flexible integration of IP cores without recompiling the application. Specifically, for IP cores such as an image signal processor, an image enhancement module, and an encoder that need to be run, whether they are new IP cores or a combination of multiple old IP cores, the corresponding information can be integrated through an XML (Extensible Markup Language) configuration file, so that the XML configuration file integrates the link layer objects to be executed, the module layer objects to be connected by the link layer objects, and the specific information of the IP cores corresponding to the module layer objects, etc.; then parameters are passed to the application to parse the XML configuration file to determine the link layer information to be run, the module information to be connected, and the device parameter information to be filled, that is, the Pipeline to be run, the Module to be connected, and the specific parameter information of the IP core, etc.

[0074] For example, taking the data transmission between the camera IP and the display IP in an application that needs to be tested as an example, assuming that it is desired to perform this test in Pipeline1, and the resolution of the camera IP is 1280×800 and the resolution of the display IP is 1920×1080, it is necessary to pre-integrate in the XML configuration file that the link layer object name is Pipeline1, the name of the first module layer object that Pipeline1 needs to connect to is camera, and the width and height of the resolution of the camera IP are set to 1280 and 800 respectively, the name of the second module layer object that Pipeline1 needs to connect to is atomic_drm, and the width and height of the resolution of the display IP are 1920 and 1080, etc.

[0075] Then, this XML configuration file is passed to the application for parsing, so as to match the link layer object to be currently run based on the field of "link layer object name", that is, the link layer information to be run includes Pipeline1; based on the "module layer object name", it is determined that the first module layer object that Pipeline1 needs to connect to is camera and the second module layer object that needs to be connected is atomic_drm, that is, the module information to be connected includes the first Module as camera and the last Module as atomic_drm, etc.; furthermore, the resolution parameters of the camera device are determined to be 1280×800 and the resolution parameters of the display device are 1920×1080 through the field of "resolution width and height", that is, the device parameter information to be filled includes the resolution parameters of the camera IP and the display IP, etc.

[0076] It can be seen that if you want to combine different IP cores or add new IP cores, you only need to modify the XML configuration, and there is no need to modify and compile the application code again, which can effectively improve the development efficiency and shorten the development cycle.

[0077] Step S20: Construct a link layer object containing a link structure based on the link layer information to be run and the module information to be connected, where the link structure is used to store the head and tail module layer object information and operation interface functions; among them, the operation interface functions include a link layer initialization function and an activation function.

[0078] Exemplarily, in this embodiment, the link layer object to be run can be determined according to the link layer information to be run, and the construction of the link layer object is performed; at the same time, a link structure corresponding to the link layer object is constructed, that is, the first module layer object and the tail module layer object that the link layer object needs to connect are determined and recorded according to the module information to be connected, and the Pipeline behavior registration for life cycle management is implemented, that is, the registration of operation interfaces pipeline_ops including initialization init, activation active, running mainloop, stopping stop, and destroying destroy is completed, that is, the operation interface functions include the link layer initialization function init, activation function active, stopping function stop, etc., to control the state of the module layer object; then, the link layer object name, the first module layer object, the tail module layer object, and the state pointers of each operation interface function are integrated to form a link structure.

[0079] It should be noted that the link structure is the core unit for modular data management and scheduling in the system. It contains a head pointer (Head Pointer) and a tail pointer (Tail Pointer), which respectively point to the first module layer object and the last module layer object in the Pipeline. With these two pointers, the Pipeline can efficiently access and manage all connected modules. In addition, the init function pointed to by the state pointer of the operation interface function in the link structure is used to initialize each module layer object connected to the Pipeline in sequence, so that all module layer objects are in the initialization state; the active function is used to activate all module layer objects to enter the main thread loop, and then start receiving and processing data; the main loop function is used for the module layer objects to transfer data through a shared buffer, etc. in the main thread loop to reduce memory copying and improve data flow efficiency; the stop function is used to terminate the main thread to stop the operation of each module layer object; the destroy function is used to recycle all resources to prevent resource leakage and ensure system stability.

[0080] Step S30: Construct a plurality of module layer objects containing module structures according to the module information to be connected and connect them to the link layer object, and create a main thread task for the module layer object. The module structure is used to store binding information, and the binding information includes the binding relationships between the module layer object, the link layer object, the device layer object, and the device callback function; wherein, the device callback function includes a device initialization function and a data processing function.

[0081] Exemplarily, in this embodiment, based on the information of the modules to be connected, the module layer objects to be created and the front-back relationship between each module layer object can be determined. Then, the module layer objects are created and the created module layer objects are connected to the corresponding link layer objects. For example, the module layer objects camera Module and atomic_drm Module are both connected to Pipeline1. It should be noted that all module layer objects can be created in parallel or sequentially according to the front-back relationship, which can be specifically determined according to actual requirements and is not limited herein.

[0082] At the same time, it is necessary to construct a corresponding module structure for each module layer object, which is the basic management unit in the Pipeline and is used to abstract different device and function modules. Each module structure specifically integrates the name of the corresponding module layer object, the module type (i.e., the IP core type, such as the IP core type being input type, output type, or input-output type), and binding information, etc. The binding information includes the data stream to be bound pointed to by the module layer object (i.e., the link layer object to be connected), the hardware device to be bound (i.e., the device layer object), and the device callback function to be bound, etc. It should be noted that the device callback function is responsible for the specific operations of the hardware device, enabling the device driver to implement the corresponding operation callbacks according to the hardware characteristics to decouple the device layer from the underlying driver, thereby achieving flexible control of hardware operations. Among them, the operations on the driver can be filled in the device callback function of the corresponding module layer object according to the specifications of the device driver to complete the configuration and implementation of the device layer, enabling the device layer to interact with the hardware by directly calling the driver interface.

[0083] Among them, the device callback functions include but are not limited to the memory type acquisition function get_memory_type, the parameter configuration function config, the device initialization function init, the data processing function handle_frame, and the destruction function deinit. The memory type acquisition function get_memory_type is used to acquire the memory type to allocate memory resources. The parameter configuration function config is used to configure device parameters to complete the initialization settings. The device initialization function init is used to initialize each device layer object, that is, to implement the initialization of the driver corresponding to the device layer object to prepare to enter the working state. The data processing function handle_frame is a callback function that runs in a loop and is used to process data frames to complete data reception or transmission. The destruction function deinit is used to de-initialize to release device resources.

[0084] It should be understood that the device callback function in this embodiment is the key to quickly expand a new IP core, that is, the hardware of the new IP core can be operated at an appropriate position in the device callback function; for example, when only the cameraModule and atomic_drm Module are integrated on the Pipeline, the images captured by the camera can be directly displayed on the screen; but if a GDC IP hardware for distortion correction needs to be added between the camera IP and the display IP, that is, a GDC Module is added between the camera Module and the atomic_drm Module, so that the images displayed on the screen will be corrected for fisheye distortion, then this embodiment only needs to add a device file to fill the callback function given by the framework, and neither the Module nor the Pipeline needs to be modified.

[0085] In addition, this embodiment also needs to create a main thread task for the module layer object. This main thread task is the core task for each module layer object to infinitely loop and process data flow, that is, the data processing code block is loop-executed through the main thread task to achieve data transfer.

[0086] Step S40: Construct a device layer object containing a device structure corresponding to the module layer object and connect it to the corresponding module layer object, and fill the device structure based on the device parameter information to be filled; among them, the link layer object controls the main thread task in the module layer object to call the target driver based on the device callback function, and realizes data transmission through the target driver, the module structure, and the device structure.

[0087] Exemplarily, in this embodiment, a device layer object corresponding to each real hardware device will also be constructed, that is, this device layer object is a virtual device of the real hardware device. For example, if the real IP core is a camera, its corresponding virtual device is the device layer object device — camera. Similarly, the device layer object corresponding to the display IP is device — display, and the device layer object corresponding to the GDC IP is device — gdc; then, according to the binding relationship, the device layer object is connected to the module layer object Module corresponding to it to form a multimedia data transmission system including three layers: the link layer, the module layer, and the device layer. It can be understood that in this embodiment, the module layer object and the device layer object are connected so that not only can the corresponding module be found from the device, but also the corresponding device can be found from the module; and the module layer object and the link layer object are connected so that not only can the link layer connecting the module be found from the module, but also the information of each module on it can be found in the link layer.

[0088] Of course, in this embodiment, it is also necessary to construct a device structure for the device layer object, and this device structure integrates fields such as the name, resolution, data format, memory, whether to apply for a buffer, and the number of buffers applied for by the corresponding device layer object. The information to be filled corresponding to the above fields can be obtained by querying the device parameter information to be filled, that is, by using the device parameter information to be filled to fill the above fields, a device structure can be formed. It should be noted that the link structure, module structure, and device structure are all formed by a general code framework, that is, only the specific information in the general code framework corresponding to the link structure, module structure, and device structure needs to be filled according to the actual data interaction and hardware device information, without the need to repeat code writing, which can effectively improve the development efficiency.

[0089] In this embodiment, when running the above multimedia data transmission system, the link layer object can control the initialization and activation of the module layer object through the head and tail module layer object information and operation interface functions stored in the link structure, so as to trigger the main thread task in the module layer object to execute the data transmission task, that is, perform target-driven initialization based on the device callback function to achieve target-driven invocation, and then implement data transmission between the IP core and the module layer object through the target drive and the device structure, and implement data transmission between module layer objects through the module structure. It can be seen that this embodiment runs the new hardware device or multiple combined hardware devices that are expected to be used in the form of a configuration file to efficiently achieve flexible integration of the IP core without recompiling the application program, and flexibly control the hardware operation through the callback function mechanism without modifying the upper-layer framework code, effectively shortening the development cycle of the multimedia data transmission system.

[0090] Further, in one embodiment, the link layer object controls the main thread task in the module layer object to call the target drive based on the device callback function through the link structure, including:

[0091] The link layer object controls each module layer object to be in an initialization state through the link layer initialization function, so as to call the target drive based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0092] Exemplarily, in this embodiment, the initialization function init of the link layer in the link structure is used to control each module layer object connected to the link layer object to be in an initialization state, so as to realize the initialization of the entire data path. Specifically, the link layer object modifies its own state to the init state, so that the module layer object also adaptively modifies its own state to the init state, so as to trigger the module layer object to call the target driver corresponding to the module layer object through the device initialization function init, and perform initialization operations on the target driver including parameters such as format, memory, and resolution. After the initialization operation of the target driver is completed, it means that the device layer object has also completed the initialization process, which further indicates that both the target driver and the device layer object are ready to start data transfer. At this time, the module layer object will notify the link layer object that it has completed the initialization operation for the link layer object to perform the next control. It should be noted that each hardware device has its corresponding device initialization function init, which is integrated into the application program in the form of a code block in advance. When needed, it can be directly called.

[0093] Further, in one embodiment, the method further includes:

[0094] Create a buffer queue for the module layer object according to the preset module type corresponding to the module layer object, and the buffer queue is used to realize data transfer between different module layer objects;

[0095] Wherein, the module structure is further used to store buffer queue information and front and back stage module information.

[0096] Exemplarily, in this embodiment, it will be controlled that data is efficiently transmitted between module layer objects through the shared buffer queue bufferqueue to avoid redundant memory copies, thereby significantly improving performance and reducing data transmission latency, and through unified buffer management, it is convenient for system maintenance and expansion. Specifically, the buffer queue is used for synchronization of the buffer between adjacent module layer objects. It is the core mechanism for data transfer and buffer synchronization between module layer objects. That is, each IP core or module needs to be classified according to the type predefined by the framework to obtain the preset module type, which is specifically one of the input type, output type, and input-output type, and a corresponding buffer queue bufferqueue is automatically created for each module layer object according to the preset module type to realize data transfer between module layer objects, so as to reduce manual intervention and have efficient data transmission and real-time guarantee.

[0097] Meanwhile, the module structure will also store the buffer queue information and the information of the previous and next level modules (i.e., the pointers pointing to the previous level Module and the next level Module), so that the module layer object can transfer data with the driver according to the buffer queue stored in the buffer queue information, and determine the transfer object of the buffer queue and transfer data according to the information of the previous and next level modules. It should be understood that whether the buffer queue contains buffers and the specific number of buffers can be defined in advance in the XML configuration file. For example, if it is set in the XML configuration file that the camera IP needs to apply for buffers and the number of buffers is 4, then 4 buffers will be created in the buffer queue of the module layer object camera Module.

[0098] It should be noted that if the communication between module layer objects is implemented by means of a message queue (i.e., using the message queue as the buffer queue), that is, using the message queue to manage the data transmission between module layer objects, that is, the input and output queues of each module layer object are designed as a message queue, so that the data between module layer objects is transmitted through the message queue, then the module structure does not need to store the information of the previous and next level modules. Among them, the message queue can provide task scheduling with different priorities and support asynchronous processing.

[0099] Specifically, when a link layer object connects to each module layer object, it will create a message queue corresponding to the function of the current module layer object. For example, the module layer object includes a video capture module (CaptureModule A), a video encoding module (GDC B), and a video playback module (Display C), and the data transfer order of the three is A - B - C. When the three are connected on a Pipeline, in the data processing function handle fame of the video capture module A, whenever a frame of video data is captured, the frame data is sent as a message to the message queue capture_to_encode for processing by the video encoding module B; the video encoding module B obtains the captured video frame from capture_to_encode, encodes the video frame, and puts the encoded video frame into another message queue encode_to_play for use by the video playback module C; the video playback module C obtains the encoded video frame from the encode_to_play queue and plays the video. It can be seen that in this solution, each module layer object does not need to care about the connection relationship of the front and back - level modules, but only needs to care about whether there are messages to be processed in the message queue that the current module layer object needs to process. For example, the video encoding module B only needs to care about whether there are messages in the capture_to_encode queue, and the video playback module C only needs to care about whether there are messages in the encode_to_play queue.

[0100] Further, in one embodiment, the module type includes an output type, an input type, and an input - output type. Creating a buffer queue corresponding to the module layer object according to the preset module type includes:

[0101] When the module type is the output type, create an output idle queue and an output busy queue as the buffer queue of the module layer object;

[0102] When the module type is the input type, create an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object;

[0103] When the module type is the input - output type, create an output idle queue, an output busy queue, an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

[0104] Exemplarily, it should be understood that different hardware devices have different input and output types. For example, the camera IP only includes an output end (i.e., the out end), which is responsible for collecting image data and transmitting it to the downstream module; the GDC IP includes both an input end (i.e., the in end) and an output end (i.e., the out end), which is mainly responsible for receiving image data, performing geometric correction, and then transmitting it to the downstream module; while the display IP only includes an input end (i.e., the in end), which is mainly responsible for receiving data and displaying it on the screen. Therefore, the camera IP is of the output type, the display IP is of the input type, and the GDC IP is of the input-output type.

[0105] Based on this, in this embodiment, different types of buffer queues will be created for the module layer objects corresponding to the hardware devices. Specifically, for the module layer object corresponding to the hardware device of the input type, this embodiment will create an output free queue out_free_queue and an output busy queue out_busy_queue as buffer queues; for the module layer object corresponding to the hardware device of the input type, an input free queue in_free_queue, an input busy queue in_busy_queue, and a waiting queue waiting_queue will be constructed as buffer queues; for the module layer object corresponding to the hardware device of the input-output type, an output free queue out_free_queue, an output busy queue out_busy_queue, an input free queue in_free_queue, an input busy queue in_busy_queue, and a waiting queue waiting_queue will be constructed as buffer queues.

[0106] And the above buffer queue information will be integrated into the module structure. That is, by assigning values to the queue fields such as out_free_queue, out_busy_queue, in_free_queue, in_busy_queue, and waiting_queue in the module structure, the creation of the buffer queue can be achieved. For example, the initial values of all queue fields are 0 to indicate that the corresponding buffer queue has not been created. If a certain queue field is assigned a value of 1, it means that the corresponding buffer queue has been created. For example, out_free_queue and out_busy_queue in the module structure of the module layer object corresponding to the camera IP will be set to 1, while in_free_queue, in_busy_queue, and waiting_queue will remain at the initial value of 0.

[0107] Furthermore, referring to Figure 3 as shown, the data transmission implemented through the target driver, the module structure, and the device structure includes:

[0108] Step S401: The link layer object controls each module layer object to be in an active state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver;

[0109] Step S402: The target driver operates on the target hardware device corresponding to the device structure based on the target buffer;

[0110] Step S403: After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the front and back module information to achieve data transmission.

[0111] Exemplarily, in this embodiment, after the entire link is initialized, the main function main will call the activation function active to change the state of the Pipeline from init to active, and then change the current state of each module layer object to "init done" (i.e., the initialization completion signal), that is, the module layer object is also in the active state; and the main thread task in each module layer object is waiting for the state of the module layer object to become "init done". When this signal is received, it means that the driver of each module layer object is ready to start transferring data, so the main thread task will start to execute the data processing function, that is, start to loop and execute the data processing code block to read the target buffer from the buffer queue of its own module layer object and submit it to the target driver, so that the target driver can perform read and write operations on the target hardware device corresponding to the device structure through the target buffer; after the target driver completes the operation on the target hardware device, the module layer object will continue to transfer the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the front and back module information to achieve data transmission between module layer objects.

[0112] It should be noted that when the module layer objects manage data transfer through the buffer queue, the processing logic from the out end of each module layer object to the in end of the next module layer object is the same; among them, for the out end of the module layer object, it first judges whether it has applied for a buffer. If it has applied for a buffer, it fills the data through the corresponding buffer queue and sends it to the in end of the next-level module layer object; if the local end has not applied for a buffer, it obtains a buffer from the buffer queue of the next-level module layer object, fills the data into the buffer and then transfers it to the next-level module layer object.

[0113] For the in - end of the module - layer object, it will check whether the buffer queue responsible for receiving buffer data is empty. If the buffer queue is not empty, it will take out the first available buffer from the queue for processing. And if this buffer is applied by the previous - level module - layer object and the current object has not applied for a buffer itself, then after completing the buffer processing, it will directly return it to the buffer queue of the previous - level module - layer object. If the taken - out buffer is applied by the module - layer object itself, it will directly process this buffer and return the buffer to the corresponding buffer queue after processing to complete the data flow. However, if the taken - out buffer is not applied by itself and the object has already applied for a buffer, it will copy the data of this buffer. After the copy is completed, it will return the copied buffer to the buffer queue of the previous - level module - layer object and return the processed buffer to the buffer queue it has applied for. Through the above buffer management method, this embodiment can achieve data sharing and synchronization between module - layer objects, avoid redundant data copy operations, improve data processing efficiency, reduce latency, and ensure the orderliness of data flow.

[0114] It should be understood that after the link layer object Pipeline becomes active, each module layer object independently executes the main thread in parallel and rotates the buffer through the buffer queue without the need to pass messages to the upper level. For example, assume that there are two module layer objects, the camera Module corresponding to the camera IP and the atomic_drmModule corresponding to the display IP, connected to Pipeline1. Since the camera Module is of the output type (i.e., output Module), it has two queues, the out free queue and the out buzy queue. And since the atomic_drm Module is of the input type (i.e., input Module), it has three queues, the in free queue, the in buzy queue, and the waiting queue. When the application reaches the main thread task of the loop of each module layer object, for the camera Module, assume that the 4 buffers it applied for are currently placed in the out free queue. Then it will take out the available free buffer from the out free queue and pass it to the corresponding camera driver for filling, and this buffer will be placed in the out buzy queue at the same time to indicate that the camera driver is processing this buffer. After the camera driver finishes processing, the application will receive a signal from the driver indicating that the processing is complete, and then the camera Module will take out the buffer from the out buzy queue and push it to the waiting queue of the next-level module layer object (i.e., the atomic_drm Module).

[0115] For the atomic_drm Module, when there is no buffer in its waiting queue, its main thread task will be in an idle state; when it detects that there is a buffer in the waiting queue, the main thread task will start to retrieve the buffer from the waiting queue and submit it to the display driver for display. At the same time, the buffer will be placed in the in buzyqueue to indicate that the display driver is using the buffer. After the display driver notifies the application that the display is complete, the atomic_drm Module will retrieve the buffer from the in buzy queue and return it to the out free queue of the previous module layer object (i.e., the camera Module), thus forming an infinite loop. It should be noted that assuming the camera Module does not apply for a buffer, an available buffer can be retrieved from the in free queue in the atomic_drm Module for data transfer, and the buffer will be returned to the in free queue in the atomic_drm Module after the data transfer is completed.

[0116] It can be seen that the camera Module only needs to continuously write data into the buffer and push it to the next-level waiting queue after finishing writing one frame. The atomic_drm Module only needs to perform its corresponding display when the waiting queue is not empty and return the buffer to the previous level after use. The two do not need to have a unified rhythm. It can be understood that each module layer object only needs to process the content in its own buffer queue without caring about the status of other module layer objects. Based on this, assuming that the camera Module takes 120 ms to write 4 buffers and the atomic_drm Module only takes 64 ms to display 4 buffers, then the performance of the atomic_drm Module can be maximized by slightly modifying the logic in the main thread task, such as making it display the buffer of the previous level.

[0117] In addition, during the above data transfer process, the module layer object does not need to receive messages from the previous level, nor does the upper layer (i.e., the link layer object) need to send messages to the device layer. The upper layer only needs to decide whether to stop the running process to end the program, thus decoupling the module layer object from the link layer object and the device layer from the link layer object.

[0118] Further, in one embodiment, reading the target buffer from the buffer queue and submitting it to the target driver includes:

[0119] The module layer object reads the target buffer from the buffer queue, and determines whether the target buffer is the buffer that it needs to process according to the target mark in the target buffer;

[0120] If yes, submitting the target buffer to the target driver;

[0121] If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object, so that the next-level module layer object executes the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0122] For example, it should be understood that for specific scenarios such as IP cores with multiple layers of output, it is necessary to use TEE plug-ins for data diversion, but the additional overhead of data diversion may affect system performance, especially in embedded systems with high real-time requirements. At the same time, developers also need to customize plug-ins to support specific business needs, and plug-in customization will increase the development cycle and cost to a certain extent.

[0123] In this embodiment, since the modules are interconnected, that is, each module can find its previous and subsequent stages, and the rotation of the buffer is realized through the buffer queue, the multi-layer output of the IP core can be realized by setting a simple logical judgment. Specifically, the module layer object can set a corresponding mark in its module structure according to the characteristics of the corresponding hardware device to characterize the buffer that it needs to process, and determine whether the same target mark exists in the received buffer to determine whether the buffer is the buffer that it needs to process; for example, for the camera IP, its corresponding camera module can set the "output resolution is 4k" mark in its module structure. When the camera module reads a buffer from the buffer queue, it will determine whether the buffer has the target mark "resolution is 4k". If it exists, it means that the buffer is the buffer that it needs to process, and then the buffer is submitted to the target driver for data flow; if it does not exist, it means that the buffer is not the buffer that it needs to process, and then the buffer is passed to the corresponding target buffer queue in the next-level module layer object for data flow.

[0124] Assume that there is a data stream where the module-level object A needs to output two layers of data, that is, one layer for the module-level object B and the other layer for the module-level object C. At this time, the three can be bound to the data stream in series to form an A-B-C structure. Then, in the data processing function handle frame thread, the module-level object A passes both layers of data to the module-level object B. When the module-level object B processes the buffer in the waiting queue, it will make some logical judgments, that is, determine whether the buffer belongs to the buffer it needs to process according to whether there is a target mark in the buffer that is the same as its own preset mark. If it belongs, it starts to process; if it does not belong, it is placed in the waiting queue of the subsequent level (i.e., the module-level object C) to achieve multi-layer output control without the need for complex framework development.

[0125] It can be seen that the data transfer mechanism in this embodiment naturally supports multi-layer output scenarios without the need to develop additional plugins or adjust the data path; among them, the link-layer object only processes the data of the current level, and other data is directly passed backward and returned layer by layer at the end to achieve flexible data flow.

[0126] In summary, this embodiment can support the integration of any IP core on the SoC with data flow requirements to achieve rapid development with low time cost; and allows flexible combination and replacement between different IP cores, and only needs to implement the device-layer callback interface, that is, developers only need to implement the corresponding callback function according to the interface specification, and do not need to modify the existing link-layer and module-layer code to complete the rapid integration of new hardware modules, avoiding the modification of the upper-layer framework code, supporting the smooth expansion of the system, and then improving the scalability and adaptability of the system to meet the needs of diverse application scenarios; in addition, by defining a standardized device-layer callback interface to support the flexible encapsulation and control of hardware modules, the integration of each hardware device becomes simple and efficient, and the decoupling of the device layer and the underlying driver is achieved through the callback function mechanism, and then the flexible control of hardware operations is realized. Based on this, this embodiment greatly improves the scalability of hardware modules and the flexibility of the system, and reduces the complexity of system maintenance and update.

[0127] In a second aspect, the embodiment of the present application further provides a multimedia data transmission system implementation device.

[0128] In one embodiment, referring to Figure 4 , Figure 4 is a schematic diagram of the functional modules of the embodiment of the multimedia data transmission system implementation device of the present application. As Figure 4 shown, the multimedia data transmission system implementation device includes:

[0129] An information parsing unit, which is used to parse a preset target XML configuration file to obtain information about the link layer to be run, information about the modules to be connected, and information about the device parameters to be filled;

[0130] A first creation unit, which is used to construct a link layer object containing a link structure based on the information about the link layer to be run and the information about the modules to be connected. The link structure is used to store information about the head and tail module layer objects and operation interface functions;

[0131] A second creation unit, which is used to construct multiple module layer objects containing module structures through the information about the modules to be connected and connect them to the link layer object, and create a main thread task for the module layer objects. The module structure is used to store binding information, and the binding information includes the binding relationships between the module layer object, the link layer object, the device layer object, and the device callback function;

[0132] A third creation unit, which is used to construct a device layer object containing a device structure corresponding to the module layer object and connect it to the corresponding module layer object, and fill the device structure based on the information about the device parameters to be filled;

[0133] Among them, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, the module structure, and the device structure.

[0134] Further, in an embodiment, the operation interface functions include a link layer initialization function and an activation function, and the device callback functions include a device initialization function and a data processing function.

[0135] Further, in an embodiment, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, including:

[0136] The link layer object controls each module layer object to be in an initialization state through the link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0137] Further, in an embodiment, the second creation unit is further used for:

[0138] Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object. The buffer queue is used to realize data transfer between different module layer objects;

[0139] Among them, the module structure is further used to store buffer queue information and information about the front and rear stage modules.

[0140] Further, in one embodiment, the data transmission implemented through target driving, module structure, and device structure includes:

[0141] The link layer object controls each module layer object to be in an active state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver.

[0142] The target driver operates on the target hardware device corresponding to the device structure based on the target buffer.

[0143] After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the front and back module information to implement data transmission.

[0144] Further, in one embodiment, the step of reading the target buffer from the buffer queue and submitting it to the target driver includes:

[0145] The module layer object reads the target buffer from the buffer queue and determines whether the target buffer is the buffer that itself needs to process according to the target mark in the target buffer.

[0146] If so, submit the target buffer to the target driver.

[0147] If not, transfer the target buffer to the corresponding target buffer queue in the next-level module layer object for the next-level module layer object to execute the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0148] Further, in one embodiment, the module type includes output type, input type, and input / output type. The creation of the buffer queue corresponding to the module layer object according to the preset module type includes:

[0149] When the module type is the output type, create an output idle queue and an output busy queue as the buffer queue of the module layer object.

[0150] When the module type is the input type, create an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

[0151] When the module type is the input / output type, create an output idle queue, an output busy queue, an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

[0152] Among them, the function realization of each unit in the multimedia data transmission system implementation device corresponds to each step in the multimedia data transmission system implementation method embodiment, and its function and implementation process will not be elaborated here one by one.

[0153] In a third aspect, an embodiment of the present application provides a multimedia data transmission system implementation device, and the multimedia data transmission system implementation device may be a device with data processing functions such as a personal computer (PC), a laptop computer, a server, etc.

[0154] Refer to Figure 5 , Figure 5 which is a schematic diagram of the hardware structure of the multimedia data transmission system implementation device involved in the embodiment of the present application. In the embodiment of the present application, the multimedia data transmission system implementation device may include a processor, a memory, a communication interface, and a communication bus.

[0155] Among them, the communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0156] The communication interface includes interfaces such as input / output (I / O) interfaces, physical interfaces, and logical interfaces for realizing the interconnection of components inside the multimedia data transmission system implementation device, and interfaces for realizing the interconnection between the multimedia data transmission system implementation device and other devices (such as other computing devices or user devices). The physical interface may be an Ethernet interface, an optical fiber interface, an ATM interface, etc.; the user device may be a display (Display), a keyboard (Keyboard), etc.

[0157] The memory may be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical memory, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0158] The processor may be a general-purpose processor, which can call the multimedia data transmission system implementation program stored in the memory and execute the multimedia data transmission system implementation method provided by the embodiments of the present application. For example, the general-purpose processor may be a central processing unit (CPU). Among them, the method executed when the multimedia data transmission system implementation program is called may refer to the various embodiments of the multimedia data transmission system implementation method of the present application, which will not be elaborated here.

[0159] Those skilled in the art can understand that Figure 5 the hardware structure shown in does not constitute a limitation to the present application, and may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0160] In a fourth aspect, embodiments of the present application further provide a computer-readable storage medium.

[0161] The multimedia data transmission system implementation program is stored on the readable storage medium of the present application. When the multimedia data transmission system implementation program is executed by a processor, the steps of the multimedia data transmission system implementation method as described above are implemented.

[0162] Among them, the method implemented when the multimedia data transmission system implementation program is executed may refer to the various embodiments of the multimedia data transmission system implementation method of the present application, which will not be elaborated here.

[0163] It should be noted that the serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0164] The terms "including" and "having" and any variations thereof in the description of the embodiments of the present application, as well as in the claims and the above-mentioned drawings, are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices. The descriptions of the terms "first", "second", "third", etc. are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit that "first", "second", and "third" are of different types.

[0165] In the description of the embodiments of the present application, terms such as "exemplary", "for example" or "for instance" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of terms such as "exemplary", "for example" or "for instance" is intended to present relevant concepts in a specific manner.

[0166] In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B. The "and / or" in the text is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0167] In some processes described in the embodiments of the present application, there are a plurality of operations or steps that appear in a specific order. However, it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or may be executed in parallel. The serial numbers of the operations are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in order or in parallel, and these operations or steps may be combined.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device to execute the methods described in the various embodiments of the present application.

[0169] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present application.

Claims

1. A method for implementing a multimedia data transmission system, characterized in that The method includes the following steps: Parse a preset target XML configuration file to obtain information of the link layer to be run, information of the modules to be connected, and information of the device parameters to be filled; Construct a link layer object including a link structure based on the information of the link layer to be run and the information of the modules to be connected, where the link structure is used to store information of the head and tail module layer objects and operation interface functions; Construct multiple module layer objects including module structures based on the information of the modules to be connected and connect them to the link layer object, and create main thread tasks for the module layer objects. The module structure is used to store binding information, and the binding information includes the binding relationships among the module layer object, the link layer object, the device layer object, and the device callback function; Construct a device layer object including a device structure corresponding to the module layer object and connect it to the corresponding module layer object, and fill the device structure based on the information of the device parameters to be filled; Among them, the link layer object controls the main thread task in the module layer object to call the target driver based on the device callback function through the link structure, and realizes data transmission through the target driver, the module structure, and the device structure; The method further includes: Create a buffer queue for the module layer object according to a preset module type corresponding to the module layer object, where the buffer queue is used to realize data flow between different module layer objects; among them, the module structure is further used to store buffer queue information and information of the previous and next level modules; The module type includes an output type, an input type, and an input / output type. Creating a buffer queue corresponding to the module layer object according to the preset module type includes: When the module type is the output type, create an output idle queue and an output busy queue as the buffer queue of the module layer object; When the module type is the input type, create an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object; When the module type is the input / output type, create an output idle queue, an output busy queue, an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

2. The method for implementing a multimedia data transmission system according to claim 1, wherein The operation interface functions include a link layer initialization function and an activation function, and the device callback functions include a device initialization function and a data processing function.

3. The method for implementing a multimedia data transmission system according to claim 2, wherein The link layer object controls the main thread task in the module layer object to call the target driver based on the device callback function through the link structure, including: The link layer object controls each module layer object to be in an initialization state through the link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

4. The method for implementing a multimedia data transmission system according to claim 2, wherein The realization of data transmission through the target driver, the module structure, and the device structure includes: The link layer object controls each module layer object to be in an activation state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read out the target buffer from the buffer queue and submit it to the target driver; The target driver operates on the target hardware device corresponding to the device structure based on the target buffer; After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the front and back-level module information to implement data transmission.

5. The method for implementing a multimedia data transmission system according to claim 4, wherein The reading of the target buffer from the buffer queue and submitting it to the target driver includes: The module layer object reads the target buffer from the buffer queue and determines whether the target buffer is the buffer that itself needs to process according to the target flag in the target buffer; If so, the target buffer is submitted to the target driver; If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object for the next-level module layer object to execute the step of reading the target buffer from the buffer queue based on the target buffer queue.

6. An apparatus for implementing a multimedia data transmission system, characterized in that, It includes: An information parsing unit, which is used to parse a preset target XML configuration file to obtain the to-be-run link layer information, the to-be-connected module information, and the to-be-filled device parameter information; A first creation unit, which is used to construct a link layer object containing a link structure based on the to-be-run link layer information and the to-be-connected module information, and the link structure is used to store the head and tail module layer object information and the operation interface function; A second creation unit, which is used to construct multiple module layer objects containing module structures through the to-be-connected module information and connect them to the link layer object, and create a main thread task for the module layer object. The module structure is used to store the binding information, and the binding information includes the binding relationships between the module layer object and the link layer object, the device layer object, and the device callback function; A third creation unit, which is used to construct a device layer object containing a device structure corresponding to the module layer object and connect it to the corresponding module layer object, and fill the device structure based on the to-be-filled device parameter information; Among them, the link layer object controls the main thread task in the module layer object to call the target driver based on the device callback function through the link structure, and realizes data transmission through the target driver, the module structure, and the device structure; The second creation unit is further used for: Creating a buffer queue for the module layer object according to the preset module type corresponding to the module layer object. The buffer queue is used to realize data flow between different module layer objects; among them, the module structure is further used to store the buffer queue information and the front and back-level module information; The module type includes an output type, an input type, and an input / output type. The creation of the buffer queue corresponding to the module layer object according to the preset module type includes: When the module type is the output type, creating an output idle queue and an output busy queue as the buffer queue of the module layer object; When the module type is the input type, creating an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object; When the module type is the input / output type, creating an output idle queue, an output busy queue, an input idle queue, an input busy queue, and a waiting queue as the buffer queue of the module layer object.

7. An apparatus for implementing a multimedia data transmission system, characterized in that, The multimedia data transmission system implementation device includes a processor, a memory, and a multimedia data transmission system implementation program stored on the memory and executable by the processor. When the multimedia data transmission system implementation program is executed by the processor, the steps of the multimedia data transmission system implementation method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that, A multimedia data transmission system implementation program is stored on the computer-readable storage medium. When the multimedia data transmission system implementation program is executed by a processor, the steps of the multimedia data transmission system implementation method according to any one of claims 1 to 5 are implemented.

Images