Streaming media data processing method and system

Abstract

The embodiment of the present invention provides a streaming media data processing method and system, which improves the performance of streaming media data reception by integrating bitstream data and message signaling into the same thread task channel according to different description file symbols and caching them into a system cache area; by enabling a packet receiving thread to obtain the bitstream data from the system cache area and store it into a streaming media internal cache area; different types of business threads read the bitstream data from the streaming media internal cache area, thereby improving the processing and forwarding performance of streaming media data, and achieving the effect of improving the receiving and forwarding speed of streaming media data.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of streaming media data processing, and in particular, to a streaming media data processing method and system. Background Art

[0002] At present, the fifth generation of mobile communication technology (5G) is gradually entering the commercial era, which has brought profound impacts and changes to the global audio and video industry, and has also made streaming media systems serving products and services such as video conferencing, video IoT, and video middleware an important mechanism for information dissemination. The quantitative change in speed brought by 5G will promote qualitative changes in the entire industry and will also help accelerate the development of streaming media systems.

[0003] In general, the streaming media systems in the industry have live broadcast, recording, playback and other main functions, and their receiving and forwarding performance has become an important indicator to measure the quality of streaming media systems. At present, the industry's receiving and forwarding speed can reach 2Gbps, but it is difficult to increase it further. In the case of high concurrency and large traffic, its processing capacity will be greatly limited: if the speed of receiving data packets is not fast enough, it will lead to system-level losses; if the speed of forwarding and processing data packets is not fast enough, it will lead to application-level losses. The receiving and forwarding performance of streaming media systems has become an important bottleneck restricting its development. Summary of the invention

[0004] The embodiment of the present invention provides a streaming media data processing method and system, so as to at least solve the problem of data packet loss in the streaming media data receiving and processing forwarding link under high traffic conditions in the related art.

[0005] According to one embodiment of the present invention, a streaming media data processing method is provided, comprising: integrating bitstream data and message signaling into the same thread task channel according to different descriptor file identifiers, and caching them into a system cache area; a packet receiving thread obtains the bitstream data from the system cache area and stores it into a streaming media internal cache area; different types of service threads read the bitstream data from the streaming media internal cache area.

[0006] In an exemplary embodiment, the code stream data and the message signaling are distinguished in the same thread task channel by a socket descriptor and a named pipe descriptor, respectively.

[0007] In an exemplary embodiment, the packet receiving thread obtains the code stream data from the system buffer area, including: the data packet structure of the code stream data is transferred using a pointer, the packet receiving thread obtains the code stream data from the system buffer area using an asynchronous IO event trigger mechanism, and adjusts the number of packets to be obtained and the maximum number of events according to the flow size of the code stream data.

[0008] In an exemplary embodiment, the packet receiving thread obtains the code stream data from the system buffer area, and further includes: using multiple packet receiving threads to obtain the code stream data from the system buffer area, and sending it to a designated CPU in a multi-core CPU for processing, and increasing the priority of the packet receiving thread.

[0009] In an exemplary embodiment, the different types of service threads read the code stream data from the internal buffer area of ​​the streaming media, including: the packet receiving thread and the service thread use the producer / consumer mode to transmit the data packets of the code stream data with the internal buffer area of ​​the streaming media as the center, wherein the packet receiving thread is the producer and the service thread is the consumer.

[0010] In an exemplary embodiment, the different types of business threads read the code stream data from the internal buffer area of ​​the streaming media, and also include: merging the different types of business threads into the same packet receiving thread and CPU processing; and selecting the packet receiving thread with the smallest load for processing according to the number of tasks of the different types of business threads.

[0011] In an exemplary embodiment, the streaming media internal buffer area is a circular resource pool.

[0012] In an exemplary embodiment, after the different types of service threads read the code stream data from the internal buffer area of ​​the streaming media, the method further includes: the different types of service threads storing the code stream data into a storage device for data persistence.

[0013] According to another embodiment of the present invention, a streaming media data processing system is provided, including: a data processing module, used to integrate code stream data and message signaling into the same thread task channel according to different descriptor file symbols, and cache them in a system cache area; a data acquisition module, used to obtain the code stream data from the system cache area and store them in a streaming media internal cache area; a data reading module, used to read the code stream data from the streaming media internal cache area according to different types of business threads.

[0014] In an exemplary embodiment, the data processing module includes at least one of the following: a first data processing unit, used to integrate the code stream data into the thread task channel according to the socket descriptor file symbol; a second data processing unit, used to integrate the message signaling into the thread task channel according to the named pipe descriptor file symbol.

[0015] In an exemplary embodiment, the data reading module includes at least one of the following: a first reading unit, which is used to read the code stream data from the internal buffer area of ​​the streaming media immediately according to the different types of the service threads after the code stream data is stored in the internal buffer area of ​​the streaming media; a second reading unit, which reads the code stream data from the internal buffer area of ​​the streaming media at the same time according to the different types of the service threads after the code stream data is stored in the internal buffer area of ​​the streaming media; and a third reading unit, which reads the code stream data from the internal buffer area of ​​the streaming media at any time according to the needs of different types of the service threads after the code stream data is stored in the internal buffer area of ​​the streaming media.

[0016] In an exemplary embodiment, it further includes: a data storage module, which is used to store the code stream data read by the data reading module into a storage device.

[0017] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0018] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0019] Through the above embodiments, the streaming media data reception performance is improved by integrating the bitstream data and message signaling into the same thread task channel according to different descriptor file symbols and caching them into the system cache area; the packet receiving thread is made to obtain the bitstream data from the system cache area and store it into the streaming media internal cache area; different types of business threads read the bitstream data from the streaming media internal cache area, thereby improving the processing and forwarding performance of the streaming media data, thereby achieving the effect of improving the receiving and forwarding speed of the streaming media data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a hardware structure block diagram of a mobile terminal of a streaming media data processing method according to an embodiment of the present invention;

[0021] Figure 2 This is a network architecture diagram for running a streaming media data processing method according to an embodiment of the present invention;

[0022] Figure 3 is a flow chart of a method for processing streaming media data according to an embodiment of the present invention;

[0023] Figure 4 is a flowchart of a service thread reading code stream data according to an embodiment of the present invention;

[0024] Figure 5 is a flow chart of a method for processing streaming media data according to an embodiment of the present invention;

[0025] Figure 6 is a structural block diagram of a streaming media data processing system according to an embodiment of the present invention;

[0026] Figure 7 is a structural block diagram of a data processing module according to an embodiment of the present invention;

[0027] Figure 8 is a structural block diagram of a data reading module according to an embodiment of the present invention;

[0028] Fig. 9 is a structural block diagram of a streaming media data processing system according to an embodiment of the present invention;

[0029] Fig.10 is a flow chart of streaming media data processing according to a scenario embodiment of the present invention;

[0030] Fig.11 is a schematic diagram of a thread management mechanism according to a scenario embodiment of the present invention;

[0031] Fig.12 It is a schematic diagram of the principle of using the vertical and horizontal packet fetching technology in the packet receiving thread according to the scenario embodiment of the present invention;

[0032] Fig.13 It is a schematic diagram of the technical principle of business thread fusion producer and consumer according to a scenario embodiment of the present invention. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0035] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a streaming media data processing method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0036] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the streaming media data processing method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] The present application embodiment can be run on Figure 2 In the network architecture shown in Figure 2As shown, the network architecture runs on a server, which can be a physical machine, a virtual machine, or a Docker containerized environment. The network architecture is divided into two parts: kernel space and user space. The kernel space refers to the environment where the operating system and driver program run. After the stream data enters the server, it will first enter the kernel space. The user space refers to the environment where the streaming media runs. The streaming media will get the stream data from the kernel space to the user space. Between the kernel space and the user space is the Socket Buffer system buffer area, which is used to realize the temporary cache of the stream data when it transitions from the kernel space to the user space. The user space includes various streaming media functional modules: the packet receiving thread RECEIVE, the streaming media internal buffer area rBuffer, the recording thread RECORD, the storage IO buffer IO Buffer, and the live broadcast thread PLAY.

[0039] In this embodiment, a streaming media data processing method running on the above mobile terminal or network architecture is provided. Figure 3 is a flow chart of a method for processing streaming media data according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0040] Step S302, integrating the bitstream data and message signaling into the same thread task channel according to different description file symbols, and buffering them into the system buffer area;

[0041] Step S304, the packet receiving thread obtains the code stream data from the system buffer area and stores it into the internal buffer area of ​​the streaming media;

[0042] Step S306: different types of service threads read the code stream data from the internal buffer area of ​​the streaming media.

[0043] Through the above steps, the streaming media data reception performance is improved by integrating the bitstream data and message signaling into the same thread task channel according to different descriptor file symbols and caching them into the system cache area; the packet receiving thread is made to obtain the bitstream data from the system cache area and store it into the streaming media internal cache area; different types of business threads read the bitstream data from the streaming media internal cache area, thereby improving the processing and forwarding performance of the streaming media data, and achieving the effect of improving the receiving and forwarding speed of the streaming media data.

[0044] The execution subject of the above steps may be a base station, a terminal, etc., but is not limited thereto.

[0045] In an exemplary embodiment, the code stream data and the message signaling are distinguished in the same thread task channel by a socket descriptor and a named pipe descriptor, respectively.

[0046] In an exemplary embodiment, the packet receiving thread obtains the code stream data from the system buffer area, including: the data packet structure of the code stream data is transferred using a pointer, the packet receiving thread obtains the code stream data from the system buffer area using an asynchronous IO event trigger mechanism, and adjusts the number of packets to be obtained and the maximum number of events according to the flow size of the code stream data.

[0047] In an exemplary embodiment, the packet receiving thread obtains the code stream data from the system buffer area, and further includes: using multiple packet receiving threads to obtain the code stream data from the system buffer area, and sending it to a designated CPU in a multi-core CPU for processing, and increasing the priority of the packet receiving thread.

[0048] In an exemplary embodiment, the different types of service threads read the code stream data from the internal buffer area of ​​the streaming media, including: the packet receiving thread and the service thread use the producer / consumer mode to transmit the data packets of the code stream data with the internal buffer area of ​​the streaming media as the center, wherein the packet receiving thread is the producer and the service thread is the consumer.

[0049] In an exemplary embodiment, Figure 4 FIG. 4 is a flowchart of a service thread reading code stream data according to an embodiment of the present invention. Figure 4 As shown, the different types of service threads read the stream data from the internal buffer area of ​​the streaming media, and also include the following steps:

[0050] Step S402, integrating the different types of service threads into the same packet receiving thread and CPU processing;

[0051] Step S404: Select the packet receiving thread with the smallest load for processing according to the number of tasks of the different types of business threads.

[0052] In an exemplary embodiment, the streaming media internal buffer area is a circular resource pool.

[0053] In an exemplary embodiment, Figure 5 is a flow chart of a method for processing streaming media data according to an embodiment of the present invention. Figure 5 As shown, after the different types of service threads read the stream data from the internal buffer area of ​​the streaming media, the process further includes: the different types of service threads store the stream data into a storage device for data persistence, that is, the process includes the following steps:

[0054] Step S502, integrating the bitstream data and message signaling into the same thread task channel according to different description file symbols, and buffering them into the system buffer area;

[0055] Step S504, the packet receiving thread obtains the code stream data from the system buffer area and stores it into the internal buffer area of ​​the streaming media;

[0056] Step S506, different types of service threads read the stream data from the internal buffer area of ​​the streaming media;

[0057] Step S508: The different types of service threads store the code stream data into a storage device for data persistence.

[0058] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0059] In this embodiment, a streaming media data processing system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0060] Figure 6 is a structural block diagram of a streaming media data processing system according to an embodiment of the present invention. Figure 6 As shown, the streaming media data processing system 60 includes: a data processing module 610, which is used to integrate the code stream data and message signaling into the same thread task channel according to different description file symbols, and cache them in the system cache area; a data acquisition module 620, which is used to obtain the code stream data from the system cache area and store them in the streaming media internal cache area; a data reading module 630, which is used to read the code stream data from the streaming media internal cache area according to different types of business threads.

[0061] In an exemplary embodiment, Figure 7 is a structural block diagram of a data processing module according to an embodiment of the present invention. Figure 7 As shown, the data processing module 70 Figure 6The data processing module 610 in the streaming media data processing system 60 shown is divided into two parts, namely: a first data processing unit 710, used to integrate the code stream data into the thread task channel according to the socket descriptor file symbol; a second data processing unit 720, used to integrate the message signaling into the thread task channel according to the named pipe descriptor file symbol.

[0062] Those skilled in the art should know that, depending on the data content, in actual implementation, the data processing module can be further subdivided into multiple data processing units, which is not limited here.

[0063] In an exemplary embodiment, Figure 8 is a structural block diagram of a data reading module according to an embodiment of the present invention. Figure 8 As shown, the data reading module 80 Figure 6 The data reading module 630 in the streaming media data processing system 60 shown is divided into three parts, namely: a first reading unit 810, which is used to read the code stream data from the streaming media internal buffer area immediately according to the different types of the business threads after the code stream data is stored in the streaming media internal buffer area; a second reading unit 820, which reads the code stream data from the streaming media internal buffer area at the same time according to the different types of the business threads after the code stream data is stored in the streaming media internal buffer area; and a third reading unit 830, which reads the code stream data from the streaming media internal buffer area at any time according to the needs of different types of the business threads after the code stream data is stored in the streaming media internal buffer area.

[0064] Those skilled in the art should know that, depending on the data content, in actual implementation, the data reading module can be further subdivided into multiple data reading units, which is not limited here.

[0065] In an exemplary embodiment, Fig. 9 is a structural block diagram of a streaming media data processing system according to an embodiment of the present invention. Fig. 9 As shown, the streaming media data processing system 90 includes Figure 6 In addition to the various modules shown in , the present invention also includes: a data storage module 910, which is used to store the code stream data read by the data reading module into a storage device.

[0066] Those skilled in the art should know that the various modules and units involved in the above embodiments can be combined together, or partially combined or integrated into one or more devices or systems as needed, as long as the corresponding functions can be achieved.

[0067] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0068] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0069] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0070] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0071] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0072] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0073] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be described below in conjunction with specific scenario embodiments.

[0074] Scenario Example 1

[0075] Fig.10 is a flow chart of streaming media data processing according to a scenario embodiment of the present invention, such as Fig.10 As shown, take the streaming media data transmission model as an example; first, establish a thread management mechanism in the whole link, propose channel convergence technology, reasonably manage and optimize threads; then, the streaming media packet receiving thread takes out the data packet from the operating system cache, and proposes vertical and horizontal packet taking technology to improve the performance of the packet receiving stage; then, the packet receiving thread stores the taken out data packet into the streaming media circular cache; then, the streaming media business thread copies the data packet from the circular cache to the related business for data processing, and proposes the fusion of producer and consumer technology to improve data forwarding performance; finally, the data is stored persistently. Next, combine Fig.10 The specific steps shown are explained in detail:

[0076] Step S1002, establishing a thread management mechanism;

[0077] Establish a thread management mechanism in the entire link of streaming media data transmission to reasonably manage and optimize threads, reduce system congestion and improve transmission performance.

[0078] Fig.11 Schematic diagram of thread management mechanism according to a scenario embodiment of the present invention. Fig.11 As shown in the figure, the data packet code stream and message signaling are integrated into the same receiving thread for processing and unified management, but they are distinguished in the entry channel. The signaling goes through the named pipe file fd, and the code stream goes through the socket fd. This avoids the use of locks to prevent multi-threaded access, reduces the investment in information management, and reduces CPU blocking and code redundancy. Thread task channel.

[0079] Step S1004, the packet receiving thread uses the vertical and horizontal packet fetching technology to receive the code stream data;

[0080] Before entering the streaming module, the media data will first reach the operating system buffer. The streaming media packet receiving thread RECEIVE is equivalent to the entry of the bitstream data, which is responsible for taking the bitstream data packet from the system buffer, that is, receiving data.

[0081] In order to improve the performance of this link, the maximum processing performance of the receiving thread is explored vertically, and the hardware and system resources are fully scheduled horizontally. The combination of vertical and horizontal is conducive to improving the performance of the packet receiving thread stage.

[0082] Fig.12 2 is a schematic diagram showing the principle of using the vertical and horizontal packet fetching technology in the packet receiving thread according to the scenario embodiment of the present invention, such as Fig.12 As shown in the figure, vertically: the data packet structure is passed using pointers to avoid large memory copies; the packet receiving thread RECEIVE adopts an asynchronous IO event trigger mechanism to adjust the number of packets and the maximum number of events according to the traffic size, thereby improving the packet fetching capability of a single thread.

[0083] Horizontally: Increase the number of receiving threads, adopt multi-threading and system multi-core CPU affinity strategy, let the packet receiving thread RECEIVE send to the specified CPU core for processing; and increase the priority of the packet receiving thread; thereby increasing the utilization of system kernel resources and giving priority to fast processing.

[0084] Step S1006, the code stream data is stored in the internal buffer area of ​​the streaming media;

[0085] After the packet receiving thread retrieves the data, it will store the data packet in the internal circular buffer rBuffer of the streaming media. rBuffer will use the resource pool and recycle it without increasing the storage capacity, thus avoiding memory fragmentation. This module is responsible for temporarily caching the data packet retrieved from the system buffer by the packet receiving thread RECEIVE. This is equivalent to backing up the data packet, so that different subsequent businesses can read the data at the same time, such as RECORD storage and PLAY forwarding, so as to achieve simultaneous storage and playback.

[0086] Step S1008, the business thread uses the fusion producer-consumer technology to process the bitstream data;

[0087] The business thread is logically located after rbuffer. Various business threads can read the data in rbuffer at the same time for their own business processes, such as RECORD thread storage, PLAY thread forwarding, and so on.

[0088] In order to improve the performance of this link, the producer-consumer mode is used to make the packet receiving thread the producer and the business thread the consumer, and these working threads are unified into the same thread to improve the performance of the rbuffer module and the data processing performance of the business thread.

[0089] Fig.13 Schematic diagram of the technical principle of business thread fusion producer and consumer according to a scenario embodiment of the present invention, such as Fig.13 As shown in the figure, rbuffer is regarded as the central point, the packet receiving thread RECEIVE is regarded as the producer, and the business threads RECORD, PLAY, etc. are regarded as consumers. The data packet is transferred from the producer RECEIVE thread to the consumer RECORD, PLAY, etc. threads through rBuffer. This technology decouples different working threads in an asynchronous calling manner, reduces the mutual influence between threads, and reduces time-consuming delays.

[0090] Based on this technology, each abstract working thread is regarded as a working task and integrated into the same actual thread and CPU for processing. At the same time, when tasks are assigned, they are assigned to the thread with the smallest load based on the number of tasks of the current thread, so that hardware resources can be more fully utilized and the extra consumption of the operating system in creating and managing threads can be reduced.

[0091] Step S1010: persistently storing the code stream data.

[0092] This link is the end of the code stream data transmission. After receiving and forwarding, it needs to be stored in a storage device for persistence. Figure 2As shown in the figure, after the recording thread RECORD reads the data packet from rBuffer, it first writes the data packet into the IOBuffer cache area. When the file is closed or the cache area is full, the cache area data is flushed to the local storage or object storage to reduce the reading and writing IO pressure.

[0093] In summary, the present invention proposes a streaming media data processing method and system. Theoretically, the data code stream can be processed at the full bandwidth speed under 10 Gbps bandwidth. The present invention greatly improves the receiving and forwarding performance of the streaming media system, and can greatly save resources and costs for audio and video related industries. In view of the problem of data packet loss in the streaming media data reception and processing forwarding link under large traffic conditions, the present invention first establishes a thread management mechanism in the whole link, integrates the code stream and signaling into the same receiving thread for processing and unified management, but distinguishes them when entering the channel, avoids the use of locks to prevent multi-threaded access, and is conducive to optimizing thread management and reducing system congestion. That is, a channel convergence technology is proposed to reasonably manage and optimize threads. Then, in the vertical direction, the packet retrieval performance of a single packet receiving thread is improved; in the horizontal direction, the kernel resources are expanded in batches, which is conducive to speeding up the retrieval of code stream data packets from the system buffer area and storing them in the internal buffer area of ​​the streaming media. That is, the vertical and horizontal packet retrieval technology is proposed to improve the performance of the data receiving stage. Finally, different working threads are decoupled by asynchronous calls to reduce the mutual influence and time-consuming delay between threads; and abstract threads are merged into the same actual threads and cores for processing, which reduces the extra consumption of threads, fully utilizes system resources, and helps speed up the data processing of the internal cache module. That is, the fusion of producer and consumer technology is proposed to improve the performance of the data processing stage.

[0094] The present invention is applicable to audio and video related industries based on streaming data processing, such as video conferencing, video IoT, video middleware, etc. Specifically, it can be used in high-concurrency scenarios with huge number of access routes and traffic, can be used for high-performance data forwarding, can be used for services without packet loss requirements, etc. The data stream link process and analysis ideas, phased performance optimization technology, and solution design of the present invention have obvious characteristics. The use of packet capture, business call chain tracking tools, and other means to achieve similar or identical purposes as the present invention should all be within the scope of protection of the present invention.

[0095] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A streaming media data processing method, characterized in that: include: Integrate the code stream data and message signaling into the same thread task channel according to different description file symbols, and cache them into the system buffer area; The packet receiving thread obtains the code stream data from the system buffer area and stores it into the internal buffer area of ​​the streaming media; Different types of service threads read the stream data from the internal buffer area of ​​the streaming media; The packet receiving thread obtains the code stream data from the system buffer area, including: the data packet structure of the code stream data is transferred using a pointer, the packet receiving thread obtains the code stream data from the system buffer area using an asynchronous IO event trigger mechanism, and adjusts the number of packets to be obtained and the maximum number of events according to the flow size of the code stream data.

2. The method according to claim 1, characterized in that The code stream data and the message signaling are distinguished in the same thread task channel by a socket descriptor and a named pipe descriptor respectively.

3. The method according to claim 1, characterized in that The packet receiving thread obtains the code stream data from the system buffer area, and further includes: Multiple packet receiving threads are used to obtain the code stream data from the system buffer area, and sent to a designated CPU in the multi-core CPU for processing, and the priority of the packet receiving thread is increased.

4. The method according to claim 3, characterized in that The different types of service threads reading the code stream data from the internal buffer area of ​​the streaming media include: The packet receiving thread and the service thread are centered on the internal buffer area of ​​the streaming media and adopt a producer / consumer mode to transmit data packets of the code stream data, wherein the packet receiving thread is the producer and the service thread is the consumer.

5. The method according to claim 4, characterized in that The different types of service threads read the code stream data from the internal buffer area of ​​the streaming media, and further include: Merging the different types of service threads into the same packet receiving thread and CPU processing; According to the number of tasks of the different types of service threads, the packet receiving thread with the smallest load is selected for processing.

6. The method according to claim 5, characterized in that The internal buffer area of ​​the streaming media is a circular resource pool.

7. The method according to claim 1, characterized in that After the different types of service threads read the code stream data from the internal buffer area of ​​the streaming media, the method further includes: The different types of service threads store the code stream data into a storage device for data persistence.

8. A streaming media data processing system, characterized in that: include: The data processing module is used to integrate the code stream data and message signaling into the same thread task channel according to different description file symbols, and cache them into the system buffer area; A data acquisition module, used to acquire the code stream data from the system buffer area and store it in the internal buffer area of ​​the streaming media; A data reading module, used for reading the code stream data from the internal buffer area of ​​the streaming media according to different types of service threads; The data acquisition module is used to acquire the code stream data from the system buffer area, including: the data packet structure of the code stream data is transferred using a pointer, an asynchronous IO event trigger mechanism is used to acquire the code stream data from the system buffer area, and the number of packets acquired and the maximum number of events are adjusted according to the flow size of the code stream data.

9. The system according to claim 8, characterized in that The data processing module includes at least one of the following: A first data processing unit, used for integrating the code stream data into the thread task channel according to the socket description file symbol; The second data processing unit is used to integrate the message signaling into the thread task channel according to the named pipe description file symbol.

10. The system according to claim 8, characterized in that The data reading module includes at least one of the following: A first reading unit, configured to read the code stream data from the internal buffer area of ​​the streaming media immediately according to different types of the service threads after the code stream data is stored in the internal buffer area of ​​the streaming media; A second reading unit, after the bitstream data is stored in the internal buffer area of ​​the streaming media, reads the bitstream data from the internal buffer area of ​​the streaming media at the same time according to different types of the service threads; The third reading unit is configured to read the code stream data from the internal buffer area of ​​the streaming media at any time according to different types of needs of the service thread after the code stream data is stored in the internal buffer area of ​​the streaming media.

11. The system according to claim 8, characterized in that Also includes: The data storage module is used to store the code stream data read by the data reading module into a storage device.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the method described in any one of claims 1 to 7 when executed by a processor.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.

Images