Multimedia data transmission system implementation method and device, equipment and storage medium

Abstract

A multimedia data transmission system implementation method, apparatus and device, and a storage medium, which relate to the field of computers, and comprise: analyzing a target XML configuration file to obtain to-be-operated link layer information, to-be-connected module information and to-be-filled device parameter information; constructing a link layer object containing a link structure body based on the to-be-operated link layer information and the to-be-connected module information; constructing a plurality of module layer objects comprising module structural bodies through the to-be-connected module information, connecting the module layer objects to the link layer objects, and creating main thread tasks for the module layer objects; constructing an equipment layer object which corresponds to the module layer object and comprises an equipment structure body, connecting the equipment layer object to the module layer object, and filling the equipment structure body based on the to-be-filled equipment parameter information; and the link layer object controls the main thread task through the link structure body to call the target driver based on the equipment callback function, and realizes data transmission through the target driver, the module structure body and the equipment structure body so as to realize flexible integration of the IP core and reduce the development period.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for implementing a multimedia data transmission system. Background Art

[0002] In the field of image processing, the raw image data collected by the sensor usually needs to be processed by multiple IP cores (Intellectual Property Core) on the SoC (System on Chip) before it can be finally passed to the terminal device for display or storage; among them, IP core refers to a reusable, verified logical function module or design unit in integrated circuit design, which includes but is not limited to image signal processor (ISP), image enhancement module, encoder, etc., which together constitute a complete image processing link. When using IP cores, sometimes a single IP core is needed, and sometimes multiple IP cores need to be combined for use, so a software framework with high flexibility, short development cycle and easy use is required.

[0003] In the related art, GStreamer is a widely used multimedia open source framework, which is usually used to build multimedia pipelines for audio, video and image processing. However, GStreamer has some limitations and shortcomings in practical applications and specific scenarios. For example, when implementing the integration of new IP cores, it is necessary to modify the existing upper-level framework code, resulting in a long development cycle; and after writing the calls of multiple IP cores into an application, if multiple IP cores need to be combined and used, it is necessary to re-modify the application code and compile and run it, which leads to a long development cycle. It can be seen that how to efficiently realize the flexible integration of IP cores to reduce the development cycle of multimedia data transmission systems is a problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for implementing a multimedia data transmission system, which can efficiently realize flexible integration of IP cores to effectively reduce the development cycle of the multimedia data transmission system.

[0005] In a first aspect, an embodiment of the present application provides a method for implementing a multimedia data transmission system, comprising the following steps: Parse the preset target XML configuration file to obtain the link layer information to be run, the module information to be connected, and the device parameter information to be filled; Constructing a link layer object including a link structure based on the link layer information to be run and the module information to be connected, wherein the link structure is used to store the first and last module layer object information and the operation interface function; Construct multiple module layer objects including module structures through the information of the modules to be connected and connect them to the link layer objects, and create a main thread task for the module layer objects, wherein the module structures are used to store binding information, and the binding information includes the binding relationship between the module layer objects and the link layer objects, the device layer objects, and the device callback functions; Constructing a device layer object including a device structure corresponding to the module layer object and connecting it to the corresponding module layer object, and filling the device structure based on the device parameter information to be filled; The link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, module structure and device structure.

[0006] In combination with the first aspect, in one implementation, the operation interface function includes a link layer initialization function and an activation function, and the device callback function includes a device initialization function and a data processing function.

[0007] In combination with the first aspect, in one implementation, the link layer object calls the target driver based on the device callback function through the main thread task in the link structure control module layer object, including: The link layer object controls each module layer object to be in an initialized state through a link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0008] In combination with the first aspect, in one implementation, the method further includes: Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object, wherein the buffer queue is used to realize data flow between different module layer objects; The module structure is also used to store buffer queue information and previous and next level module information.

[0009] In combination with the first aspect, in one implementation, the data transmission is realized through the target driver, the module structure and the device structure, including: The link layer object controls each module layer object to be in an activated state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver; The target driver operates the target hardware device corresponding to the device structure based on the target buffer; After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the previous and next-level module information to achieve data transmission.

[0010] In combination with the first aspect, in one implementation, reading the target buffer from the buffer queue and submitting it to the target driver includes: The module layer object reads the target buffer from the buffer queue, and determines whether the target buffer is the buffer that needs to be processed by itself according to the target mark in the target buffer; If yes, submitting the target buffer to the target driver; If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object, so that the next-level module layer object executes the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0011] In combination with the first aspect, in one implementation, the module type includes an output type, an input type, and an input-output type, and the creating a buffer queue corresponding to the module layer object according to the preset module type includes: When the module type is an output type, an output idle queue and an output busy queue are created as buffer queues of the module layer object; When the module type is an input type, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object; When the module type is an input-output type, an output idle queue, an output busy queue, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object.

[0012] In a second aspect, an embodiment of the present application provides a multimedia data transmission system implementation device, including: An information parsing unit, which is used to parse a preset target XML configuration file to obtain link layer information to be run, module information to be connected, and device parameter information to be filled; A first creation unit, which is used to construct a link layer object including a link structure based on the link layer information to be run and the module information to be connected, wherein the link structure is used to store the first and last module layer object information and the operation interface function; A second creation unit is used to construct a plurality of module layer objects including module structures through information of modules to be connected and connect them to link layer objects, and to create a main thread task for the module layer objects, wherein the module structures are used to store binding information, and the binding information includes binding relationships between the module layer objects and link layer objects, device layer objects, and device callback functions; A third creation unit, which is used to construct a device layer object including a device structure corresponding to the module layer object and connect to the corresponding module layer object, and fill the device structure based on the device parameter information to be filled; The link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, module structure and device structure.

[0013] In combination with the second aspect, in one implementation, the operation interface function includes a link layer initialization function and an activation function, and the device callback function includes a device initialization function and a data processing function.

[0014] In conjunction with the second aspect, in one implementation, the link layer object calls the target driver based on the device callback function by the main thread task in the link structure control module layer object, including: The link layer object controls each module layer object to be in an initialized state through a link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0015] In conjunction with the second aspect, in one implementation, the second creating unit is further configured to: Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object, wherein the buffer queue is used to realize data flow between different module layer objects; The module structure is also used to store buffer queue information and previous and next level module information.

[0016] In conjunction with the second aspect, in one implementation, the data transmission is realized through the target driver, the module structure and the device structure, including: The link layer object controls each module layer object to be in an activated state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver; The target driver operates the target hardware device corresponding to the device structure based on the target buffer; After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the previous and next-level module information to achieve data transmission.

[0017] In combination with the second aspect, in one implementation, reading the target buffer from the buffer queue and submitting it to the target driver includes: The module layer object reads the target buffer from the buffer queue, and determines whether the target buffer is the buffer that needs to be processed by itself according to the target mark in the target buffer; If yes, submitting the target buffer to the target driver; If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object, so that the next-level module layer object executes the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0018] In conjunction with the second aspect, in one implementation, the module type includes an output type, an input type, and an input-output type, and the creating a buffer queue corresponding to the module layer object according to the preset module type includes: When the module type is an output type, an output idle queue and an output busy queue are created as buffer queues of the module layer object; When the module type is an input type, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object; When the module type is an input-output type, an output idle queue, an output busy queue, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object.

[0019] In the third aspect, an embodiment of the present application provides a multimedia data transmission system implementation device, which includes a processor, a memory, and a multimedia data transmission system implementation program stored in the memory and executable by the processor, wherein when the multimedia data transmission system implementation program is executed by the processor, the steps of the multimedia data transmission system implementation method as described above are implemented.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a multimedia data transmission system implementation program is stored. When the multimedia data transmission system implementation program is executed by a processor, the steps of the multimedia data transmission system implementation method as described above are implemented.

[0021] The beneficial effects brought by the technical solution provided by the embodiments of the present application include: The target XML configuration file is parsed to obtain the link layer information to be run, the module information to be connected and the device parameter information to be filled, and a link layer object containing a link structure is constructed based on the link layer information to be run and the module information to be connected; then multiple module layer objects containing module structures are constructed through the module information to be connected and connected to the link layer object, and a main thread task is created for the module layer object, and then a device layer object containing a device structure corresponding to the module layer object is constructed and connected to the corresponding module layer object to form a layered architecture, and the device structure is filled based on the device parameter information to be filled; wherein, the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, the module structure and the device structure. It can be seen that the present application runs the new hardware device or multiple combined hardware devices that are desired to be used through the configuration file, so as to efficiently realize the flexible integration of the IP core without recompiling the application, and realizes the flexible control of the hardware operation through the callback function mechanism without modifying the upper framework code, which effectively shortens the development cycle of the multimedia data transmission system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of an embodiment of a method for implementing a multimedia data transmission system of the present application; Figure 2 A schematic diagram of a multimedia data transmission system according to an embodiment of the present application; Figure 3 For this application Figure 1 A detailed flow chart of step S40; Figure 4 A schematic diagram of functional modules of an embodiment of a device for implementing a multimedia data transmission system of the present application; Figure 5 A schematic diagram of the hardware structure of a multimedia data transmission system implementation device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] First, some technical terms in the present application are explained to facilitate those skilled in the art to understand the present application.

[0025] API (Application Programming Interface): Application programming interface.

[0026] GDC (Geometric Distortion Correctio): Geometric distortion correction.

[0027] TEE (T-distributed Estimator of Error): It is the tee command used in Linux system to represent T-shaped structure, which is similar to the diversion function of pipeline, that is, sending input to multiple output locations at the same time.

[0028] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0029] In a first aspect, an embodiment of the present application provides a method for implementing a multimedia data transmission system.

[0030] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the multimedia data transmission system implementation method of the present application. Figure 1 As shown, the multimedia data transmission system implementation method includes: Step S10: Parse the preset target XML configuration file to obtain the link layer information to be run, the module information to be connected and the device parameter information to be filled.

[0031] Exemplarily, it should be understood that currently, multimedia data transmission systems are often constructed through GStreamer, but the architecture and API design of GStreamer are relatively complex, and the entry threshold is high, that is, developers need to master a large number of plug-ins, pipelines, elements, and data flow management details to effectively develop and optimize applications, and when integrating new IP cores, beginners must deeply understand the internal architecture of GStreamer and adapt it under its API restrictions, resulting in a long development and debugging cycle. It should be noted that the IP core in this embodiment refers to a real hardware device.

[0032] See also Figure 2As shown, in this embodiment, in order to effectively reduce the development difficulty and development cycle, a conceptually simple layered architecture consisting of a link layer Pipeline, a module layer Module, and a device layer Device will be constructed to provide a flexible and scalable data processing architecture, which aims to efficiently manage the data flow between multiple IP cores or modules in the SoC, thereby ensuring the efficient transmission and processing of data between different modules. Among them, the link layer Pipeline is mainly responsible for the series connection and scheduling between the module layers, so as to realize the orderly flow of data flow through a unified Pipeline structure; the module layer Module is mainly responsible for abstracting the hardware device, that is, each module layer encapsulates the specific functional logic of its corresponding hardware device, etc., for processing and transmitting data, but the actual hardware operation is completed by the device driver device_driver; the device layer Device is directly associated with the specific hardware device, so as to be responsible for calling the driver to complete the actual transmission, processing and hardware operation of the data.

[0033] In addition, this embodiment will run the new hardware device or multiple combinations of hardware devices that you want to use through the configuration file, so as to efficiently realize the flexible integration of IP cores without recompiling the application. Specifically, for the IP cores such as image signal processors, image enhancement modules, encoders, etc. that need to be run, whether they are new IP cores or a combination of multiple old IP cores, the corresponding information can be integrated through the XML (Extensible Markup Language) configuration file, so that the XML configuration file integrates the link layer object that needs to be executed, the module layer object to be connected to the link layer object, and the specific information of the IP core corresponding to the module layer object; then pass the parameters to the application to parse the XML configuration file to determine the link layer information to be run, the module information to be connected, and the device parameter information to be filled, that is, the Pipeline to be run and the Module to be connected and the specific parameter information of the IP core.

[0034] For example, take the case where an application needs to test the data transmission from the camera IP to the display IP. Assuming that you want to perform the test in Pipeline1, and the resolution of the camera IP is 1280×800 and the resolution of the display IP is 1920×1080, you need to integrate the link layer object name as Pipeline1 in the XML configuration file in advance, the first module layer object name that Pipeline1 needs to connect to is camera, and the width and height of the camera IP resolution are set to 1280 and 800 respectively, the second module layer object name that Pipeline1 needs to connect to is atomic_drm, and the width and height of the display IP resolution are 1920 and 1080, etc.

[0035] The XML configuration file is then passed to the application for parsing, so as to match the link layer object that currently needs to be run as Pipeline1 according to the "link layer object name" field, that is, the link layer information to be run includes Pipeline1; based on the "module layer object name", it is determined that the first module layer object that Pipeline1 needs to connect to is camera and the second module layer object that needs to be connected is atomic_drm, that is, the module information to be connected includes the first Module as camera and the tail Module as atomic_drm, etc.; and then through the "resolution width and height" field, it is determined that the resolution parameters of the camera device are 1280×800 and the resolution parameters of the display device are 1920×1080, that is, the device parameter information to be filled in includes the resolution parameters of the camera IP and the display IP, etc.

[0036] It can be seen that if you want to combine different IP cores or add new IP cores, you only need to modify the XML configuration without re-modifying and compiling the application code, which can effectively improve development efficiency and shorten the development cycle.

[0037] Step S20: construct a link layer object including a link structure based on the link layer information to be run and the module information to be connected, wherein the link structure is used to store the first and last module layer object information and the operation interface function; wherein the operation interface function includes a link layer initialization function and an activation function.

[0038] Exemplarily, in this embodiment, the link layer object that needs to be run can be determined according to the link layer information to be run and the link layer object can be constructed; at the same time, a link structure corresponding to the link layer object is constructed, that is, the first module layer object and the tail module layer object that the link layer object needs to connect to are determined and recorded according to the module information to be connected, and the Pipeline behavior registration for life cycle management is implemented, that is, the registration of the operation interface pipeline_ops including initialization init, activation active, mainloop, stop stop and destruction destroy is completed, that is, the operation interface function includes the link layer initialization function init, activation function active, stop function stop, etc., to control the status of the module layer object; then the above-mentioned link layer object name, the first module layer object, the tail module layer object and the status pointers of each operation interface function are integrated to form a link structure.

[0039] It should be noted that the link structure is the core unit for modular data management and scheduling in the system. It contains a head pointer and a tail pointer, which point to the first module layer object and the last module layer object in the Pipeline respectively. With these two pointers, the Pipeline can efficiently access and manage all connected modules. In addition, the init function pointed to by the state pointer of the operation interface function in the link structure is used to initialize each module layer object connected to the Pipeline in turn, even if all module layer objects are in the initialization state; the active function is used to activate all module layer objects so that they enter the main thread loop and start receiving and processing data; the main loop function is used for module layer objects to transfer data through shared buffers in the main thread loop to reduce memory copies and improve data flow efficiency; the stop function is used to terminate the main thread to stop the operation of each module layer object; the destroy function is used to recycle all resources to prevent resource leakage and ensure system stability.

[0040] Step S30: Construct multiple module layer objects including module structures through the information of the modules to be connected and connect them to the link layer objects, and create a main thread task for the module layer objects. The module structure is used to store binding information, and the binding information includes the binding relationship between the module layer objects and the link layer objects, the device layer objects and the device callback function; wherein the device callback function includes a device initialization function and a data processing function.

[0041] Exemplarily, in this embodiment, based on the information of the module to be connected, the module layer objects that need to be created and the front-end and back-end relationships between the module layer objects can be determined, and then the module layer objects are created and connected to the corresponding link layer objects, such as connecting the module layer objects camera Module and atomic_drm Module to Pipeline1; it should be noted that all module layer objects can be created in parallel or in sequence according to the front-end and back-end relationships, which can be determined according to actual needs and is not limited here.

[0042] At the same time, it is necessary to build a corresponding module structure for each module layer object, which is the basic management unit in Pipeline and is used to abstract different devices and functional modules; each module structure specifically integrates the name of the corresponding module layer object, the module type (i.e., IP core type, such as the IP core type is input type, output type, or input-output type) and binding information, etc. The binding information includes the data stream to be bound pointed to by the module layer object (i.e., the link layer object to be connected), the hardware device to be bound (i.e., the device layer object) and the device callback function to be bound. It should be noted that the device callback function is responsible for the specific operation of the hardware device, that is, the device driver implements the corresponding operation callback according to the hardware characteristics to decouple the device layer and the underlying driver, thereby realizing flexible control of the hardware operation; among them, the operation of the driver can be filled in the device callback function of the corresponding module layer object according to the specification of the device driver to complete the configuration and implementation of the device layer, so that the device layer can interact with the hardware by directly calling the driver interface.

[0043] Among them, the device callback function includes but is not limited to the memory type acquisition function get_memory_type, the parameter configuration function config, the device initialization function init, the data processing function handle_frame and the destruction function deinit; the memory type acquisition function get_memory_type is used to obtain the memory type to allocate memory resources; the parameter configuration function config is used to configure the device parameters to complete the initialization settings; the device initialization function init is used to initialize each device layer object, that is, to implement the initialization of the driver corresponding to the device layer object to prepare to enter the working state; the data processing function handle_frame is a cyclically running callback function, which is used to process data frames to complete data reception or transmission; the destruction function deinit is used to deinitialize to release device resources.

[0044] It should be understood that the device callback function in this embodiment is the key to quickly expand a new IP core, that is, to operate the hardware of the new IP core at the appropriate position in the device callback function; for example, when only cameraModule and atomic_drm Module are integrated on the Pipeline, the image captured by the camera can be displayed directly on the screen; but if it is necessary to add a distortion correction GDC IP hardware between the camera IP and the display IP, that is, add a GDC Module between the camera Module and the atomic_drm Module so that the image displayed on the screen will be corrected for fisheye distortion, then this embodiment only needs to add a device file to fill the callback function given by the framework, and neither the Module nor the Pipeline needs to be modified.

[0045] In addition, this embodiment also needs to create a main thread task for the module layer object. The main thread task is the core task of each module layer object to process data flow in an infinite loop, that is, the data processing code block is executed in a loop through the main thread task to realize data transmission.

[0046] Step S40: Construct a device layer object corresponding to the module layer object and including a device structure, connect it to the corresponding module layer object, and fill the device structure based on the device parameter information to be filled; wherein the link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, module structure and device structure.

[0047] For example, in this embodiment, a corresponding device layer object will be constructed for each real hardware device, that is, the device layer object is a virtual device of the real hardware device. For example, if the real IP core is a camera, its corresponding virtual device is the device layer object device. — camera, similarly, the device layer object corresponding to the display IP is device — display, the device layer object corresponding to GDCIP is device — gdc; then, according to the binding relationship, the device layer object is connected to the corresponding module layer object Module to form a multimedia data transmission system including a link layer, a module layer and a device layer. It can be understood that, in this embodiment, the module layer object is connected to the device layer object so that not only the corresponding module can be found from the device, but also the corresponding device can be found from the module; and the module layer object is connected to the link layer object so that not only the link layer connected to the module can be found from the module, but also the information of each module on it can be found in the link layer.

[0048] Of course, this embodiment also needs to build a device structure for the device layer object, and the device structure integrates the name, resolution, data format, memory, whether to apply for a buffer and the number of buffers applied for the corresponding device layer object, and the information to be filled in the above fields can be obtained by querying the device parameter information to be filled, that is, the above fields are filled with the device parameter information to be filled, and the device structure can be formed. It should be noted that the link structure, module structure and device structure are all formed by a general code framework, that is, it is only necessary to fill in the specific information in the general code framework corresponding to the link structure, module structure and device structure according to the actual data interaction and hardware device information, without the need to repeatedly write code, which can effectively improve development efficiency.

[0049] In this embodiment, when the above-mentioned multimedia data transmission system is running, the link layer object can control the initialization and activation of the module layer object through the first and last module layer object information and operation interface functions stored in the link structure, so as to trigger the main thread task in the module layer object to perform the data transmission task, that is, to initialize the target driver based on the device callback function to realize the call of the target driver, and then realize the data transmission between the IP core and the module layer object through the target driver and the device structure, and realize the data transmission between the module layer objects through the module structure. It can be seen that this embodiment runs the new hardware device or multiple combined hardware devices that are desired to be used through the configuration file, so as to efficiently realize the flexible integration of the IP core without recompiling the application program, and realizes the flexible control of the hardware operation through the callback function mechanism without modifying the upper framework code, which effectively shortens the development cycle of the multimedia data transmission system.

[0050] Further, in one embodiment, the link layer object calls the target driver based on the device callback function through the main thread task in the link structure control module layer object, including: The link layer object controls each module layer object to be in an initialized state through a link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0051] Exemplarily, in this embodiment, the link layer initialization function init in the link structure controls each module layer object connected to the link layer object to be in the initialization state, so as to realize the initialization of the entire data path; specifically, the link layer object modifies its own state to the init state, so that the module layer object also adaptively modifies its own state to the init state, so as to trigger the module layer object to call the target driver corresponding to the module layer object through the device initialization function init, and perform initialization operations on the target driver including parameters such as format, memory, and resolution; after the initialization operation of the target driver is completed, it means that the device layer object has also completed the initialization processing, which further indicates that the target driver and the device layer object are ready to start the data flow accurately, and at this time the module layer object will notify the link layer object that it has completed the initialization operation, so that the link layer object can perform the next step of control. It should be noted that each hardware device has its corresponding device initialization function init, which is pre-integrated into the application in the form of a code block, and can be directly called when it is needed.

[0052] Furthermore, in one embodiment, the method further includes: Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object, wherein the buffer queue is used to realize data flow between different module layer objects; The module structure is also used to store buffer queue information and previous and next level module information.

[0053] Exemplarily, in this embodiment, the control module layer objects are used to efficiently transfer data through a shared buffer queue bufferqueue to avoid redundant memory copies, thereby significantly improving performance and reducing data transmission delays, and facilitating system maintenance and expansion through unified buffer management. Specifically, the buffer queue is used to synchronize buffers between adjacent module layer objects, which is the core mechanism for data transfer and buffer synchronization between module layer objects, that is, each IP core or module needs to be classified according to the predefined types of the framework to obtain a preset module type, which is specifically one of the input type, output type, and input-output type, and automatically creates a corresponding buffer queue buffer queue for each module layer object according to the preset module type to realize data flow and transmission between module layer objects, so as to reduce manual intervention and have efficient data transmission and real-time guarantee.

[0054] At the same time, the module structure will also store the buffer queue information and the previous and next module information (i.e., the pointer to the previous module and the pointer to the next module), so that the module layer object can transfer data with the driver according to the buffer queue stored in the buffer queue information, and determine the transfer object of the buffer queue according to the previous and next module information and transfer data. It should be understood that whether the buffer queue contains buffer buffers and the specific number of buffers can be defined in the XML configuration file in advance. For example, if the camera IP is set in the XML configuration file to apply for buffers and the number of buffers is 4, 4 buffers will be created in the buffer queue of the module layer object camera Module.

[0055] It should be noted that if the communication between module-level objects is realized by means of message queues (that is, message queues are used as buffer queues), that is, message queues are used to manage data transmission between module-level objects, that is, the input and output queues of each module-level object are designed as a message queue, so that data between module-level objects is transmitted through message queues, then the module structure does not need to store the information of the previous and next modules. Among them, the message queue can provide task scheduling of different priorities and support asynchronous processing.

[0056] Specifically, each time the link layer object connects to a module layer object, it will create a message pair corresponding to the function of the current module layer object. For example, the module layer object includes a video capture module (CaptureModule A), a video encoding module (GDC B) and a video playback module (Display C), and the data transmission order of the three is ABC, and the three are connected to a Pipeline. Then, in the data processing function handle fame of the video capture module A, whenever a frame of video data is captured, the frame data is sent as a message to the message queue capture_to_encode for processing by the video encoding module B; the video encoding module B obtains the captured video frame from capture_to_encode, encodes the video frame, and puts the encoded video frame into another message queue encode_to_play for use by the video playback module C; the video playback module C obtains the encoded video frame from the encode_to_play queue and plays the video. It can be seen that in this scheme, each module layer object does not need to care about the connection relationship between the previous and next level modules, but only needs to care whether there are pending messages in the message queue that the current module layer object needs to process. For example, the video encoding module B only needs to care whether there are messages in the capture_to_encode queue, while the video playback module C only needs to care whether there are messages in encode_to_play.

[0057] Further, in one embodiment, the module type includes an output type, an input type, and an input-output type, and the creating a buffer queue corresponding to the module layer object according to the preset module type includes: When the module type is an output type, an output idle queue and an output busy queue are created as buffer queues of the module layer object; When the module type is an input type, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object; When the module type is an input-output type, an output idle queue, an output busy queue, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object.

[0058] For example, it should be understood that different hardware devices have different input and output types. For example, the camera IP only includes the output end (i.e., the out end), which is responsible for collecting image data and passing it to the downstream module; the GDC IP includes both the input end (i.e., the in end) and the output end (i.e., the out end), which is mainly responsible for receiving image data and passing it to the downstream module after geometric correction; and the display IP only includes the input end (i.e., the in end), which is mainly responsible for receiving data and displaying it on the screen; therefore, the camera IP is of the output type, the display IP is of the input type, and the GDC IP is of the input-output type.

[0059] Based on this, this embodiment will create different types of buffer queues for the module layer objects corresponding to the hardware devices. Specifically, for the module layer objects corresponding to the hardware devices whose module type is the input type, this embodiment will create the output free queue out_free_queue and the output busy queue out_busy_queue as buffer queues; for the module layer objects corresponding to the hardware devices whose module type is the input type, the input free queue in_free_queue, the input busy queue in_busy_queue and the waiting queue waiting_queue will be constructed as buffer queues; for the module layer objects corresponding to the hardware devices whose module type is the input and output type, the output free queue out_free_queue, the output busy queue out_busy_queue, the input free queue in_free_queue, the input busy queue in_busy_queue and the waiting queue waiting_queue will be constructed as buffer queues.

[0060] The above buffer queue information will be integrated into the module structure, that is, by assigning values ​​to queue fields such as out_free_queue, out_busy_queue, in_free_queue, in_busy_queue and waiting_queue in the module structure, the creation of the buffer queue can be realized; for example, the initial value of all queue fields is 0, indicating that the corresponding buffer queue has not been created. If a queue field is assigned a value of 1, it indicates that the corresponding buffer queue has been created. For example, the out_free_queue and out_busy_queue in the module structure of the module layer object corresponding to the camera IP will be set to 1, while in_free_queue, in_busy_queue and waiting_queue will remain at the initial value of 0.

[0061] Further, see Figure 3 As shown, the data transmission is realized through the target driver, the module structure and the device structure, including: Step S401: the link layer object controls each module layer object to be in an activated state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver; Step S402: the target driver operates the target hardware device corresponding to the device structure based on the target buffer; Step S403: After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the previous-level module information to achieve data transmission.

[0062] Exemplarily, in this embodiment, after the initialization of the entire link is completed, the main function main will call the activation function active to change the state of the Pipeline from init to active, and then change the current state of each module layer object to "init done" (i.e., the initialization completion signal), that is, the module layer object is also in the active state; and the main thread task in each module layer object is waiting for the state of the module layer object to change to "init done". When the signal is received, it means that the driver of each module layer object is ready to start transferring data, and the main thread task will start to execute the data processing function, that is, start to loop the execution of the data processing code block to read the target buffer from the buffer queue of the module layer object where it is located and submit it to the target driver, so that the target driver can perform operations such as reading and writing through the target buffer and the target hardware device corresponding to the device structure; after the target driver completes the operation on the target hardware device, the module layer object will continue to pass the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object through the previous and next-level module information to realize data transmission between module layer objects.

[0063] It should be noted that when module layer objects manage data transmission through the buffer queue, the out end of each module layer object to the in end of the next module layer object uses the same processing logic; among them, for the out end of the module layer object, it first determines whether it has applied for the buffer buffer. If it has applied for the buffer buffer, it fills the data through the corresponding buffer queue and sends it to the in end of the next module layer object; if the local end has not applied for the buffer buffer, it obtains the buffer buffer from the buffer queue of the next module layer object, fills the data into the buffer buffer and then passes it to the next module layer object.

[0064] For the in end of the module layer object, it will check whether the buffer queue responsible for receiving the buffer data is empty. If the buffer queue is not empty, the first available buffer is taken out from the queue for processing. If the buffer is applied by the previous module layer object and the object has not applied for a buffer, it will be directly returned to the buffer queue of the previous module layer object after completing the buffer processing. If the taken out buffer is applied by the module layer object itself, the buffer is directly processed and returned to the corresponding buffer queue after processing to complete the data flow. However, if the taken out buffer is not applied by itself and the object has applied for a buffer, the data is copied to the buffer, and after the copy is completed, the copied buffer is returned to the buffer queue of the previous module layer object, and the processed buffer is returned to the buffer queue applied by itself. This embodiment can realize data sharing and synchronization between module layer objects through the above buffer management method, avoid redundant data copy operations, improve data processing efficiency, reduce delays, and ensure the orderliness of data flow.

[0065] It should be understood that after the link layer object Pipeline is activated, each module layer object independently executes the main thread in parallel and rotates the buffer through the buffer queue without passing messages to the upper level. For example, suppose that Pipeline1 is connected to two module layer objects, camera Module corresponding to the camera IP and atomic_drmModule corresponding to the display IP, and because camera Module is of output type (ie, output Module), it has two queues, out free queue and out buzy queue, and atomic_drm Module is of input type (ie, input Module), it has three queues, in free queue, in buzy queue and waiting queue; when the application goes to the main thread task of the respective module layer object loop, for camera Module, assuming that the 4 buffers applied for by itself are currently placed in the out free queue, it will take out the free buffer that it can currently use from the outfree queue and pass it to the corresponding camera driver to fill it, and the buffer will be placed in the out buzyqueue at the same time to indicate that the camera driver is processing the buffer; after the camera driver has finished processing, the application will receive a processing completion signal from the driver to know that this frame of data has been processed, then the camera Module will take the buffer from the out buzy queue. The queue is taken out and passed to the waiting queue of the next-level module layer object (i.e. atomic_drm Module). For atomic_drm Module, when there is no buffer in its waiting queue, its main thread task will be in an idle state; when it detects that there is a buffer in the waiting queue, the main thread task will start to take out the buffer from the waiting queue and submit it to the display driver for display, and at the same time put the buffer in the in buzyqueue to indicate that the display driver is using the buffer; after the display driver informs the application that the display is complete, the atomic_drm Module will take the buffer out of the in buzy queue and return it to the out free queue of the previous module layer object (ie, camera Module), thus forming an infinite loop. It should be noted that, assuming that the cameraModule has not applied for a buffer, the available buffer can be taken out of the in free queue in the atomic_drm Module for data transfer, and the buffer will be returned to the in freequeue in the atomic_drm Module after the data transfer is completed.

[0066] It can be seen that the camera Module only needs to write data into the buffer continuously, and push it to the waiting queue of the next level after writing one, while the atomic_drm Module only needs to do its corresponding display when the waiting queue is not empty, and return the buffer to the previous level after use. The two do not need a unified rhythm; it is understandable that each module layer object only needs to process the contents in its own buffer queue without caring about the status of other module layer objects. Based on this, assuming that the camera Module takes 120ms to write 4 buffers, and the atomic_drm Module only needs 64ms to display 4 buffers, then you only need to slightly modify the logic in the main thread task to maximize the performance of the atomic_drmModule, such as letting it display the buffer of the previous level.

[0067] In addition, during the above data flow process, the module layer object does not need to receive messages from the upper level, nor does it need the upper layer (i.e., the link layer object) to send messages to the device layer. The upper layer only needs to decide whether to stop the running process to end the program, thereby decoupling the module layer object from the link layer object and the device layer from the link layer object.

[0068] Further, in one embodiment, the step of reading the target buffer from the buffer queue and submitting the target buffer to the target driver includes: The module layer object reads the target buffer from the buffer queue, and determines whether the target buffer is the buffer that needs to be processed by itself according to the target mark in the target buffer; If yes, submitting the target buffer to the target driver; If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object, so that the next-level module layer object executes the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0069] For example, it should be understood that for specific scenarios such as IP cores with multiple layers of output, it is necessary to use TEE plug-ins for data diversion, but the additional overhead of data diversion may affect system performance, especially in embedded systems with high real-time requirements. At the same time, developers also need to customize plug-ins to support specific business needs, and plug-in customization will increase the development cycle and cost to a certain extent.

[0070] In this embodiment, since the modules are interconnected, that is, each module can find its previous and subsequent stages, and the rotation of the buffer is realized through the buffer queue, the multi-layer output of the IP core can be realized by setting a simple logical judgment. Specifically, the module layer object can set a corresponding mark in its module structure according to the characteristics of the corresponding hardware device to characterize the buffer that it needs to process, and determine whether the same target mark exists in the received buffer to determine whether the buffer is the buffer that it needs to process; for example, for the camera IP, its corresponding camera module can set the "output resolution is 4k" mark in its module structure. When the camera module reads a buffer from the buffer queue, it will determine whether the buffer has the target mark "resolution is 4k". If it exists, it means that the buffer is the buffer that it needs to process, and then the buffer is submitted to the target driver for data flow; if it does not exist, it means that the buffer is not the buffer that it needs to process, and then the buffer is passed to the corresponding target buffer queue in the next-level module layer object for data flow.

[0071] Assume that there is a data stream for module layer object A that needs to output two layers of data, one layer to module layer object B and the other layer to module layer object C. At this time, the three can be bound to the data stream in series to form an ABC structure. Then in the data processing function handle frame thread, module layer object A passes both layers of data to module layer object B. When module layer object B processes the buffer in the waiting queue, it will make some logical judgments, that is, determine whether the buffer belongs to the buffer it wants to process based on whether there is a target tag in the buffer that is the same as its own preset tag. If it does, it will start processing, and if it does not, it will be placed in the waiting queue of the next level (that is, module layer object C) to achieve multi-layer output control without the need for complex framework development.

[0072] It can be seen that the data transmission mechanism in this embodiment naturally supports multi-layer output scenarios without the need to develop additional plug-ins or adjust the data path; among them, the link layer object only processes the data at the current layer, and other data is directly passed backward and returned layer by layer at the end point, thereby realizing flexible data flow.

[0073] In summary, this embodiment can support the integration of any IP core with data flow requirements on the SoC to achieve rapid development with low time cost; and allow flexible combination and replacement between different IP cores, and only need to implement the device layer callback interface, that is, the developer only needs to implement the corresponding callback function according to the interface specification, and does not need to modify the existing link layer and module layer code to complete the rapid integration of the new hardware module, avoiding the modification of the upper-layer framework code, supporting the smooth expansion of the system, and then improving the scalability and adaptability of the system to meet the needs of diverse application scenarios; in addition, by defining a standardized device layer callback interface to support the flexible encapsulation and control of hardware modules, the integration of various hardware devices becomes simple and efficient, and the device layer and the underlying driver are decoupled through the callback function mechanism, and then the flexible control of hardware operations is realized. Based on this, this embodiment greatly improves the scalability of the hardware module and the flexibility of the system, and reduces the complexity of system maintenance and updates.

[0074] In a second aspect, an embodiment of the present application also provides a device for implementing a multimedia data transmission system.

[0075] In one embodiment, referring to Figure 4 , Figure 4 This is a functional module diagram of an embodiment of a multimedia data transmission system implementation device of the present application. Figure 4 As shown, the multimedia data transmission system implementation device includes: An information parsing unit, which is used to parse a preset target XML configuration file to obtain link layer information to be run, module information to be connected, and device parameter information to be filled; A first creation unit, which is used to construct a link layer object including a link structure based on the link layer information to be run and the module information to be connected, wherein the link structure is used to store the first and last module layer object information and the operation interface function; A second creation unit is used to construct a plurality of module layer objects including module structures through information of modules to be connected and connect them to link layer objects, and to create a main thread task for the module layer objects, wherein the module structures are used to store binding information, and the binding information includes binding relationships between the module layer objects and link layer objects, device layer objects, and device callback functions; A third creation unit, which is used to construct a device layer object including a device structure corresponding to the module layer object and connect to the corresponding module layer object, and fill the device structure based on the device parameter information to be filled; The link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, module structure and device structure.

[0076] Furthermore, in one embodiment, the operation interface function includes a link layer initialization function and an activation function, and the device callback function includes a device initialization function and a data processing function.

[0077] Further, in one embodiment, the link layer object calls the target driver based on the device callback function through the main thread task in the link structure control module layer object, including: The link layer object controls each module layer object to be in an initialized state through a link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

[0078] Furthermore, in one embodiment, the second creating unit is further configured to: Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object, wherein the buffer queue is used to realize data flow between different module layer objects; The module structure is also used to store buffer queue information and previous and next level module information.

[0079] Further, in one embodiment, the data transmission is realized through the target driver, the module structure and the device structure, including: The link layer object controls each module layer object to be in an activated state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver; The target driver operates the target hardware device corresponding to the device structure based on the target buffer; After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the previous and next-level module information to achieve data transmission.

[0080] Further, in one embodiment, the step of reading the target buffer from the buffer queue and submitting the target buffer to the target driver includes: The module layer object reads the target buffer from the buffer queue, and determines whether the target buffer is the buffer that needs to be processed by itself according to the target mark in the target buffer; If yes, submitting the target buffer to the target driver; If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object, so that the next-level module layer object executes the step of reading the target buffer from the buffer queue based on the target buffer queue.

[0081] Further, in one embodiment, the module type includes an output type, an input type, and an input-output type, and the creating a buffer queue corresponding to the module layer object according to the preset module type includes: When the module type is an output type, an output idle queue and an output busy queue are created as buffer queues of the module layer object; When the module type is an input type, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object; When the module type is an input-output type, an output idle queue, an output busy queue, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object.

[0082] Among them, the functional implementation of each unit in the above-mentioned multimedia data transmission system implementation device corresponds to the various steps in the above-mentioned multimedia data transmission system implementation method embodiment, and its functions and implementation processes are no longer repeated here.

[0083] In a third aspect, an embodiment of the present application provides a multimedia data transmission system implementation device, which may be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0084] Reference Figure 5 , Figure 5 The hardware structure diagram of the multimedia data transmission system implementation device involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the multimedia data transmission system implementation device may include a processor, a memory, a communication interface, and a communication bus.

[0085] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0086] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect devices within multimedia data transmission systems, and to interconnect devices with other devices (such as other computing devices or user devices) in multimedia data transmission systems. Physical interfaces can be Ethernet interfaces, optical fiber interfaces, ATM interfaces, etc.; user devices can be display screens (Display), keyboards (Keyboard), etc.

[0087] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0088] The processor may be a general-purpose processor, and the general-purpose processor may call the multimedia data transmission system implementation program stored in the memory and execute the multimedia data transmission system implementation method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the multimedia data transmission system implementation program is called may refer to the various embodiments of the multimedia data transmission system implementation method of the present application, and will not be described in detail here.

[0089] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0090] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0091] The readable storage medium of the present application stores a multimedia data transmission system implementation program, wherein when the multimedia data transmission system implementation program is executed by a processor, the steps of the multimedia data transmission system implementation method as described above are implemented.

[0092] Among them, the method implemented when the multimedia data transmission system implementation program is executed can refer to the various embodiments of the multimedia data transmission system implementation method of the present application, and will not be repeated here.

[0093] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0094] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0095] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0096] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0097] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned implementation methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0099] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for implementing a multimedia data transmission system, characterized in that: The following steps are involved: Parse the preset target XML configuration file to obtain the link layer information to be run, the module information to be connected, and the device parameter information to be filled; Constructing a link layer object including a link structure based on the link layer information to be run and the module information to be connected, wherein the link structure is used to store the first and last module layer object information and the operation interface function; Construct multiple module layer objects including module structures through the information of the modules to be connected and connect them to the link layer objects, and create a main thread task for the module layer objects, wherein the module structures are used to store binding information, and the binding information includes the binding relationship between the module layer objects and the link layer objects, the device layer objects, and the device callback functions; Constructing a device layer object including a device structure corresponding to the module layer object and connecting it to the corresponding module layer object, and filling the device structure based on the device parameter information to be filled; The link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, module structure and device structure.

2. The method for implementing a multimedia data transmission system according to claim 1, characterized in that: The operation interface function includes a link layer initialization function and an activation function, and the device callback function includes a device initialization function and a data processing function.

3. The method for implementing a multimedia data transmission system according to claim 2, characterized in that: The link layer object calls the target driver based on the device callback function through the main thread task in the link structure control module layer object, including: The link layer object controls each module layer object to be in an initialized state through a link layer initialization function, so as to call the target driver based on the device initialization function and perform initialization processing to complete the initialization of the device layer object.

4. The method for implementing a multimedia data transmission system according to claim 2, characterized in that: The method further comprises: Creating a buffer queue for the module layer object according to a preset module type corresponding to the module layer object, wherein the buffer queue is used to realize data flow between different module layer objects; The module structure is also used to store buffer queue information and previous and next level module information.

5. The method for implementing a multimedia data transmission system according to claim 4, characterized in that: The data transmission is realized through the target driver, the module structure and the device structure, including: The link layer object controls each module layer object to be in an activated state according to the activation function, so that the main thread task in the module layer object executes the data processing function to read the target buffer from the buffer queue and submit it to the target driver; The target driver operates the target hardware device corresponding to the device structure based on the target buffer; After the operation is completed, the module layer object transfers the target buffer to the corresponding buffer queue in the next-level module layer object or the buffer queue in the previous-level module layer object based on the previous and next-level module information to achieve data transmission.

6. The method for implementing a multimedia data transmission system according to claim 5, characterized in that: The step of reading the target buffer from the buffer queue and submitting it to the target driver includes: The module layer object reads the target buffer from the buffer queue, and determines whether the target buffer is the buffer that it needs to process according to the target mark in the target buffer; If yes, submitting the target buffer to the target driver; If not, the target buffer is transferred to the corresponding target buffer queue in the next-level module layer object, so that the next-level module layer object executes the step of reading the target buffer from the buffer queue based on the target buffer queue.

7. The method for implementing a multimedia data transmission system according to claim 4, characterized in that: The module type includes an output type, an input type, and an input-output type. The step of creating a buffer queue corresponding to a module layer object according to a preset module type includes: When the module type is an output type, an output idle queue and an output busy queue are created as buffer queues of the module layer object; When the module type is an input type, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object; When the module type is an input-output type, an output idle queue, an output busy queue, an input idle queue, an input busy queue and a waiting queue are created as buffer queues of the module layer object.

8. A multimedia data transmission system implementation device, characterized in that: include: An information parsing unit, which is used to parse a preset target XML configuration file to obtain link layer information to be run, module information to be connected, and device parameter information to be filled; A first creation unit, which is used to construct a link layer object including a link structure based on the link layer information to be run and the module information to be connected, wherein the link structure is used to store the first and last module layer object information and the operation interface function; A second creation unit is used to construct a plurality of module layer objects including module structures through information of modules to be connected and connect them to link layer objects, and to create a main thread task for the module layer objects, wherein the module structures are used to store binding information, and the binding information includes binding relationships between the module layer objects and link layer objects, device layer objects, and device callback functions; A third creation unit, which is used to construct a device layer object including a device structure corresponding to the module layer object and connect to the corresponding module layer object, and fill the device structure based on the device parameter information to be filled; The link layer object controls the main thread task in the module layer object through the link structure to call the target driver based on the device callback function, and realizes data transmission through the target driver, module structure and device structure.

9. A multimedia data transmission system implementation device, characterized in that: The multimedia data transmission system implementation device includes a processor, a memory, and a multimedia data transmission system implementation program stored in the memory and executable by the processor, wherein when the multimedia data transmission system implementation program is executed by the processor, the steps of the multimedia data transmission system implementation method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a multimedia data transmission system implementation program, wherein when the multimedia data transmission system implementation program is executed by a processor, the steps of the multimedia data transmission system implementation method according to any one of claims 1 to 7 are implemented.

Images