Live Streaming Data Processing Methods and Platforms

Abstract

This application discloses a live streaming data processing method and platform, relating to the field of video processing technology. It includes: a streaming media data acquisition system acquiring live streaming data stored in a streaming media server, and adding an acquisition timestamp to each acquired live streaming data; the streaming media data acquisition system simultaneously delivering the live streaming data with the added timestamp to a message queue and a first data storage system; a first data processing system consuming data from the message queue and storing the consumed data in a second data storage system; a second data processing system consuming the delivered data and storing the consumed data in the first data storage system; and the first and second data processing systems updating the on-time of the live stream matching the live stream name stored in the consumed data based on the live stream name and acquisition timestamp. This application ensures the availability of the live streaming data.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a live data processing method and platform. Background Technology

[0002] During live streaming, to facilitate processing of the latest live streaming data at the business level, a typical approach involves collecting the data from the streaming media server using an acquisition system. This data is then stored in a message queue component. Subsequently, business-level processing can be performed on the live streaming data stored in the message queue. This method decouples the data acquisition and data storage components for the latest live streaming data.

[0003] However, if the message queue component fails, the business layer will no longer be able to process the latest live streaming data. Summary of the Invention

[0004] In view of this, a live streaming data processing method and platform are provided to solve the above problems.

[0005] This application provides a live streaming data processing platform, which includes a streaming media server, a streaming media data acquisition system, a message queue, a first data processing system, a second data processing system, a first data storage system, and a second data storage system, wherein:

[0006] The streaming media server is used to store the live streaming data of the broadcaster;

[0007] The streaming media data acquisition system is used to acquire the live streaming data from the streaming media server, and add an acquisition timestamp to the acquired live streaming data after each acquisition.

[0008] The streaming media data acquisition system is also used to deliver live streaming data with acquisition timestamps to the message queue, and to deliver live streaming data with acquisition timestamps to the first data storage system through a preset delivery method.

[0009] The first data processing system is used to consume data from the message queue, store the consumed data in the second data storage system, and update the push time of the live stream that matches the first live stream name stored in the second data storage system according to the first live stream name and the collection timestamp contained in the consumed data.

[0010] The second data processing system is used to consume data delivered through the preset delivery method, store the consumed data in the first data storage system, and update the push time of the live stream that matches the second live stream name stored in the first data storage system according to the second live stream name and the collection timestamp contained in the consumed data.

[0011] Optionally, the live streaming data processing platform further includes a data aggregation system, which is used for:

[0012] The push time of each live stream is obtained from the first data storage system and the second data storage system;

[0013] The latest push time of each live stream is determined based on the push time of each live stream obtained from the first data storage system and the second data storage system.

[0014] Optionally, the streaming media data acquisition system includes a data acquisition instruction sending module, a data acquisition module, and a timestamp addition module, wherein:

[0015] A data acquisition instruction sending module is used to send a data acquisition instruction to the streaming media server. The data acquisition instruction carries a third live stream name of the live stream to be acquired. After receiving the data acquisition instruction, the streaming media server transmits the live streaming data matching the third live stream name to the streaming media data acquisition system.

[0016] The data acquisition module is used to collect live streaming data transmitted by the streaming media server;

[0017] The timestamp addition module is used to add a timestamp to the live streaming data after the data acquisition module acquires a single live streaming data.

[0018] Optionally, the timestamp addition module is further configured to obtain the NTP time after the data acquisition module acquires each live streaming data, and add the NTP time as the timestamp to the acquired live streaming data.

[0019] Optionally, the preset delivery method is a remote procedure call, and the streaming media data acquisition system is also used to deliver live streaming data with acquisition timestamps to the first data storage system via a remote procedure call.

[0020] Optionally, both the first data storage system and the second data storage system are Redis databases.

[0021] Optionally, the streaming media server may have multiple servers, and the streaming media data acquisition system may also be used to acquire live streaming data of all broadcasters who are in a live streaming state from the multiple streaming media servers.

[0022] This application also provides a live streaming data processing method, applied in a live streaming data processing platform, the live streaming data processing platform including a streaming media server, a streaming media data acquisition system, a message queue, a first data processing system, a second data processing system, a first data storage system, and a second data storage system, the method comprising:

[0023] The streaming media data acquisition system collects the live streaming data of the anchor stored in the streaming media server, and adds a collection timestamp to the collected live streaming data after each collection.

[0024] The streaming media data acquisition system delivers live streaming data with acquisition timestamps to the message queue, and the streaming media data acquisition system delivers live streaming data with acquisition timestamps to the first data storage system through a preset delivery method;

[0025] The first data processing system consumes data from the message queue and stores the consumed data in the second data storage system. The first data processing system also updates the push time of the live stream that matches the first live stream name stored in the second data storage system based on the first live stream name and the collection timestamp contained in the consumed data.

[0026] The second data processing system consumes the data delivered through the preset delivery method and stores the consumed data in the first data storage system. The second data processing system also updates the push time of the live stream that matches the second live stream name stored in the first data storage system based on the second live stream name and the collection timestamp contained in the consumed data.

[0027] Optionally, the live streaming data processing platform further includes a data aggregation system, and the method further includes:

[0028] The data aggregation system obtains the push time of each live stream from the first data storage system and the second data storage system;

[0029] The data aggregation system determines the latest push time of each live stream based on the push time of each live stream obtained from the first data storage system and the second data storage system.

[0030] Optionally, the streaming media data acquisition system collects the live streaming data of the broadcaster stored in the streaming media server, and adds a collection timestamp to the collected live streaming data after each piece of live streaming data is collected, including:

[0031] The streaming media data acquisition system sends a data acquisition instruction to the streaming media server. The data acquisition instruction carries the third live stream name of the live stream to be acquired. After receiving the data acquisition instruction, the streaming media server transmits the live push data that matches the third live stream name to the streaming media data acquisition system.

[0032] The streaming media data acquisition system collects the live streaming data transmitted by the streaming media server;

[0033] After each live streaming data acquisition system acquires a live streaming data, it adds a timestamp to the acquired live streaming data.

[0034] The live streaming data processing platform in this application achieves dual-link production of the latest live streaming data by transmitting the streaming media data collected by the streaming media data acquisition system through dual data links. This ensures high availability and stability of the latest live streaming data even if the message queue component fails. Furthermore, the platform updates the latest push time of the live stream when storing it in both the first and second data storage systems. Therefore, when the latest live streaming data is needed, the platform can determine which data storage system to store it in based on the push time stored in both systems. This allows for timely retrieval of the latest live streaming data when processing is required. Attached Figure Description

[0035] Figure 1 This is a schematic diagram illustrating the application environment of one embodiment of the live data processing method according to this application.

[0036] Figure 2 This is a framework diagram of one embodiment of the live streaming data processing platform described in this application;

[0037] Figure 3 This is a flowchart of one embodiment of the live data processing method described in this application. Detailed Implementation

[0038] The advantages of this application are further illustrated below with reference to the accompanying drawings and specific embodiments.

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0040] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0041] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0042] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0043] The following is a definition of the terminology used in this application:

[0044] Live stream: The transmission of live audio and video data, which can be transmitted to viewers over a network as a stable and continuous stream.

[0045] Live Streaming Room: In the live streaming service, each streamer has a unique identifier. Based on the identifier, you can enter the streamer's live streaming viewing page, which is the streamer's room.

[0046] TCP is a connection-oriented, reliable, byte-stream-based transport layer communication protocol.

[0047] Message queue: A container that stores messages during message transmission.

[0048] RPC: Remote Procedure Call.

[0049] Figure 1This illustration shows an application scenario provided by an embodiment of this application, which includes a server 100, a broadcaster terminal 200, and a viewer terminal 300. The server 100, broadcaster terminal 200, and viewer terminal 300 are located in a wireless network or a wired network, and the server 100 can interact with the broadcaster terminal 200 and the viewer terminal 300. In some embodiments, the number of broadcaster terminals 200 (…) Figure 1 Only one is shown in the image) and the number of 300 viewer terminals ( Figure 1 (Only one is shown in the image) This is not a limitation. The viewer terminal 300 and the broadcaster terminal 200 can also use the server 100 and CDN (Content Delivery Network) as transmission intermediaries and realize data interaction between them through the Internet.

[0050] In this embodiment, server 100 can be a standalone server or a server cluster; it can be a local server or a cloud server. The specific server type is not limited in this embodiment. The broadcast terminal 200 and the viewer terminal 300 can be mobile phones, smartphones, laptops, desktop computers, tablets, personal digital assistants (PDAs), media players, smart TVs, wearable electronic devices, etc. The specific terminal device type is not limited in this embodiment.

[0051] In this embodiment, a live streaming client may be installed in the broadcaster terminal 200, which is used to enable the broadcaster to conduct live video streaming. The live streaming client may be a computer application (APP) installed on the broadcaster terminal 200, or it may be a web client, which may refer to an application developed based on the web architecture.

[0052] In this embodiment, a viewer client may be installed in the viewer terminal 300. This viewer client is used to enable viewers to receive and display live content. The viewer client may be a computer application (APP) installed on the viewer terminal 300, or it may be a web client, which may refer to an application developed based on a web architecture.

[0053] The following will provide several embodiments within the aforementioned exemplary application environment to illustrate the live streaming data processing platform of this application. See also... Figure 2The live streaming data processing platform provided in this embodiment includes: a streaming media server 21, a streaming media data acquisition system 22, a message queue 23, a first data processing system 24, a second data processing system 25, a first data storage system 26, and a second data storage system 27.

[0054] Streaming media server 21 is used to store the live streaming data of the broadcaster.

[0055] Specifically, when a streamer is broadcasting live, the live streaming data generated during the broadcast will be pushed to the streaming media server 21 through the streamer's client so that the streaming media server 21 can store the live streaming data.

[0056] The live streaming data refers to the live audio and video data pushed by the broadcaster client to the streaming media server 21. The live streaming data includes the live stream name (stream ID).

[0057] It should be noted that, since the streamer will continuously generate live streaming data during the live broadcast, in this embodiment, this continuous live streaming data can be regarded as a single live stream.

[0058] In one exemplary implementation, to enable broadcasters located throughout the country to conduct live broadcasts, streaming media servers 21 can be deployed nationwide. When broadcasting, broadcasters can select appropriate streaming media servers for streaming according to preset strategies.

[0059] It should be noted that, in order to distinguish the live streaming data from different broadcasters, the streaming media server 21 assigns a live stream name (also known as a stream ID) to each live stream before the broadcaster pushes their live streaming data to the streaming media server 21. When the broadcaster's terminal pushes the live streaming data generated during the current broadcast to the streaming media server 21, it adds the live stream name to each live streaming data. In this way, after receiving the live streaming data, the streaming media server 21 can determine which live stream the received live streaming data belongs to based on the live stream name in the live streaming data.

[0060] The streaming media data acquisition system 22 is used to acquire the live streaming data from the streaming media server 21, and add an acquisition timestamp to the acquired live streaming data after each acquisition.

[0061] Specifically, in order to ensure that the streamer's viewers can watch the streamer's live video in a timely manner, in this embodiment, the streaming media data acquisition system 22 collects the streamer's live streaming data from the streaming media server. Furthermore, each time the streaming media data acquisition system 22 collects a piece of live streaming data from the streaming media server, it adds a collection timestamp to the collected live streaming data.

[0062] The collection timestamp is the system time of the collected live streaming data. For example, if live streaming data 'a' was collected at 11:20:00 on November 7, 2023, then the time 11:20:00 on November 7, 2023 will be added to live streaming data 'a'.

[0063] In one exemplary implementation, when there are multiple streaming media servers 21, the streaming media data acquisition system 21 will collect live streaming data of all broadcasters in a live streaming state from the multiple streaming media servers 21.

[0064] Specifically, if the streaming media server 21 includes streaming media server a, streaming media server b and streaming media server c, then the streaming media data acquisition system 21 will collect the live streaming data of each broadcaster in a live streaming state stored in streaming media server a, streaming media server b and streaming media server c respectively.

[0065] In one exemplary embodiment, the streaming media data acquisition system 22 includes a data acquisition instruction sending module, a data acquisition module, and an acquisition timestamp adding module.

[0066] The data acquisition instruction sending module is used to send a data acquisition instruction to the streaming media server 21. The data acquisition instruction carries the third live stream name of the live stream to be acquired. After receiving the data acquisition instruction, the streaming media server 21 transmits the live streaming data matching the third live stream name to the streaming media data acquisition system.

[0067] Specifically, when the streaming media data acquisition system 22 needs to collect the live streaming data of the anchor from the streaming media service 21, the streaming media data acquisition system 22 will call the data acquisition instruction sending module to send a data acquisition instruction to the streaming media server 21.

[0068] It should be noted that, since the streaming media server 21 stores live streaming data corresponding to different live streams, in this embodiment, when the data acquisition instruction sending module sends a data acquisition instruction to the streaming media server 21, it will include the third live stream name of the live stream to be acquired in the data acquisition instruction. Thus, when the streaming media server 21 receives the data acquisition instruction, it will find the live streaming data of the live stream matching the third live stream name carried in the data acquisition instruction. After finding the live streaming data, the streaming media server 21 will transmit the live streaming data to the streaming media data acquisition system 22 via the network.

[0069] The data acquisition module is used to acquire live streaming data transmitted by the streaming media server 21.

[0070] Specifically, when the streaming media server 21 transmits the live streaming data to the streaming media data acquisition system 22 via the network, the streaming media data acquisition system 22 will call the data acquisition module to collect the live streaming data transmitted by the streaming media server 21.

[0071] The timestamp addition module is used to add a timestamp to the live streaming data after the data acquisition module acquires a single live streaming data.

[0072] Specifically, after the data acquisition module acquires a live streaming data, the streaming media data acquisition system 22 will call the acquisition timestamp addition module to add an acquisition timestamp to the acquired live streaming data.

[0073] In one exemplary embodiment, the timestamp addition module is further configured to obtain the NTP time after the data acquisition module acquires each live streaming data, and add the NTP time as the timestamp to the acquired live streaming data.

[0074] Specifically, NTP (Network Time Protocol) time is the system time obtained from the NTP server.

[0075] The streaming media data acquisition system 22 is also used to deliver live streaming data with acquisition timestamps to the message queue, and to deliver live streaming data with acquisition timestamps to the first data storage system through a preset delivery method.

[0076] In this embodiment, after the data acquisition module acquires each piece of live streaming data, the acquisition timestamp addition module obtains the current NTP time from the NTP server, and after obtaining the NTP time, adds the NTP time as the acquisition timestamp to the acquired live streaming data.

[0077] The streaming media data acquisition system 22 is also used to deliver live streaming data with acquisition timestamps to the message queue 23, and to deliver live streaming data with acquisition timestamps to the first data storage system 26 through a preset delivery method.

[0078] Specifically, after adding the collection timestamp to the collected live streaming data, the streaming media data acquisition system 22 will deliver the live streaming data with the added collection timestamp to the message queue 23. At the same time, the streaming media data acquisition system 22 will also deliver the live streaming data with the added collection timestamp to the first data storage system 26 through another link in a preset delivery method.

[0079] The message queue can be Kafka, ActiveMQ, RabbitMQ, RocketMQ, etc.

[0080] The preset delivery method is a pre-defined delivery method, which can be a remote procedure call, an HTTP request, or the like.

[0081] In an exemplary embodiment, when the preset delivery method can be a remote procedure call, the streaming media data acquisition system 22, after adding the acquisition timestamp to the acquired live streaming data, will deliver the live streaming data with the added acquisition timestamp to the first data storage system 26 via a remote procedure call.

[0082] In this embodiment, by simultaneously sending the live streaming data with the added collection timestamp to the message queue 23 and the first data storage system 26, dual writing of the live streaming data is achieved, ensuring the high availability and stability of the live streaming data.

[0083] The first data processing system 24 is used to consume data from the message queue 23, store the consumed data in the second data storage system 27, and update the push time of the live stream that matches the first live stream name stored in the second data storage system 27 according to the first live stream name and the collection timestamp contained in the consumed data.

[0084] Specifically, the first data processing system 24 can be a Flink system, a Spark system, or the like. The first data processing system 24 consumes data from the message queue 23, and after consuming each piece of data, it parses the data to extract the first live stream name and the collection timestamp contained within it.

[0085] The first live stream name is used to determine which live stream the data belongs to. The collection timestamp is used to determine when the data was collected.

[0086] In this embodiment, after consuming data from the message queue, the first data processing system 24 stores the consumed data in the second data storage system 27 to achieve persistent storage of the live stream data and facilitate the subsequent distribution of the live stream data to viewers.

[0087] After obtaining the first live stream name and collection timestamp contained in the consumed data, the first data processing system 24 will search for the push time of the live stream that matches the first live stream name in the second data storage system 27 based on the first live stream name, and then update the push time with the collection timestamp.

[0088] The "pushing time" refers to the time used to determine whether the live stream is still being pushed.

[0089] In this embodiment, when storing the live streaming data into the second data storage system 27, the push time is updated according to the collection timestamp, so that the push time recorded in the second data storage system 27 is the latest push time of the live stream.

[0090] The second data processing system 25 is used to consume the data delivered through the preset delivery method, store the consumed data in the first data storage system 26, and update the push time of the live stream that matches the second live stream name stored in the first data storage system 26 according to the second live stream name and the collection timestamp contained in the consumed data.

[0091] Specifically, the second data processing system 25 can also be a Flink system, a Spark system, etc. The second data processing system 25 consumes the data delivered through the preset delivery method, and after consuming each piece of data, it parses the data to extract the second live stream name and the collection timestamp contained in the data.

[0092] The second live stream name is used to determine which live stream the data belongs to. The collection timestamp is used to determine when the data was collected.

[0093] In this embodiment, after consuming the data, the second data processing system 25 stores the consumed data in the first data storage system 26 to achieve persistent storage of the live stream data.

[0094] After obtaining the second live stream name and collection timestamp contained in the consumed data, the second data processing system 25 will search for the push time of the live stream that matches the second live stream name in the first data storage system 26 based on the second live stream name, and then update the push time with the collection timestamp.

[0095] The "pushing time" refers to the time used to determine whether the live stream is still being pushed.

[0096] In this embodiment, when storing live streaming data into the first data storage system 26, the push time is updated according to the collection timestamp, so that the push time recorded in the first data storage system 26 is the latest push time of the live stream.

[0097] In an exemplary implementation, in order to quickly obtain the live streaming data stored in the first data storage system 26 and the second data storage system 27, the first data storage system 26 and the second data storage system 27 are preferably Redis databases.

[0098] The live streaming data processing platform in this application achieves dual-link production of the latest live streaming data by transmitting the streaming media data collected by the streaming media data acquisition system through dual data links. This ensures high availability and stability of the latest live streaming data even if the message queue component fails. Furthermore, the platform updates the latest push time of the live stream when storing it in both the first and second data storage systems. Therefore, when the latest live streaming data is needed, the platform can determine which data storage system to store it in based on the push time stored in both systems. This allows for timely retrieval of the latest live streaming data when processing is required.

[0099] In one exemplary embodiment, to facilitate subsequent provision of query services to external parties, the live streaming data processing platform further includes a data aggregation system.

[0100] The data aggregation system is used to obtain the push time of each live stream from the first data storage system 26 and the second data storage system 27; and to determine the latest push time of each live stream based on the push time of each live stream obtained from the first data storage system and the second data storage system.

[0101] Specifically, the data aggregation system retrieves the push times of all live streams stored in the first data storage system 26 and the second data storage system 27 in real time. After obtaining the push times of all live streams, it compares the push times of the same live stream recorded in the first data storage system 26 and the second data storage system 27, and takes the later push time as the latest push time of the current live stream. For example, for live stream 1, the push time stored in the first data storage system 26 is 14:16:20 on November 4, 2023, and the push time stored in the second data storage system 27 is 14:16:15 on November 4, 2023. Then, the data aggregation system will take the time 14:16:15 on November 4, 2023 as the latest push time of live stream 1.

[0102] In this embodiment, the data is aggregated and processed through a data aggregation system. This way, when a query service is subsequently provided to the outside world, it can be directly determined which storage system's live streaming data should be provided for different stream names.

[0103] Reference Figure 3 This is a live streaming data processing method according to an embodiment of this application. The method is applied to a live streaming data processing platform. The live streaming data processing platform includes a streaming media server, a streaming media data acquisition system, a message queue, a first data processing system, a second data processing system, a first data storage system, and a second data storage system. The live streaming data processing platform described in the above embodiment is the same as the one described above.

[0104] In this embodiment, the method includes:

[0105] Step S30: The streaming media data acquisition system collects the live streaming data of the anchor stored in the streaming media server, and adds a collection timestamp to the collected live streaming data after each piece of live streaming data is collected.

[0106] Step S31: The streaming media data acquisition system delivers the live streaming data with the acquisition timestamp added to the message queue, and the streaming media data acquisition system delivers the live streaming data with the acquisition timestamp added to the first data storage system through a preset delivery method.

[0107] Step S32: The first data processing system consumes data from the message queue and stores the consumed data in the second data storage system. The first data processing system also updates the push time of the live stream that matches the first live stream name stored in the second data storage system according to the first live stream name and the collection timestamp contained in the consumed data.

[0108] Step S33: The second data processing system consumes the data delivered through the preset delivery method and stores the consumed data in the first data storage system. The second data processing system also updates the push time of the live stream that matches the second live stream name stored in the first data storage system according to the second live stream name and the collection timestamp contained in the consumed data.

[0109] In one exemplary embodiment, the live streaming data processing platform further includes a data aggregation system, and the method further includes: the data aggregation system obtaining the push time of each live stream from the first data storage system and the second data storage system; and the data aggregation system determining the latest push time of each live stream based on the push time of each live stream obtained from the first data storage system and the second data storage system.

[0110] In one exemplary embodiment, the streaming media data acquisition system collects live streaming data of broadcasters stored in the streaming media server, and adds a collection timestamp to the collected live streaming data after each data point is collected. This includes: the streaming media data acquisition system sending a data acquisition instruction to the streaming media server, the data acquisition instruction carrying a third live stream name to be collected; the streaming media server transmitting live streaming data matching the third live stream name to the streaming media data acquisition system after receiving the data acquisition instruction; the streaming media data acquisition system collecting the live streaming data transmitted by the streaming media server; and the streaming media data acquisition system adding a collection timestamp to the collected live streaming data after each data point is collected.

[0111] In one exemplary implementation, the streaming media data acquisition system obtains the NTP time after acquiring each live streaming data, and adds the NTP time as the acquisition timestamp to the acquired live streaming data.

[0112] In one exemplary implementation, the preset delivery method is a remote procedure call (RPC) method, in which the streaming media data acquisition system delivers live streaming data with acquisition timestamps to the first data storage system via a RPC.

[0113] In one exemplary implementation, both the first data storage system and the second data storage system are Redis databases.

[0114] In one exemplary implementation, the streaming media servers are multiple, and the streaming media data acquisition system collects live streaming data from all the broadcasters who are in a live streaming state from the multiple streaming media servers.

[0115] The live streaming data processing method in this application achieves dual-link production of the latest live streaming data by transmitting the streaming media data collected by the streaming media data acquisition system through dual data links. This ensures high availability and stability of the latest live streaming data even if the message queue component fails. Furthermore, the live streaming data processing platform in this application updates the latest push time of the live stream when storing it in both the first and second data storage systems. Therefore, when the latest live streaming data is needed, the push time in both systems can be used to determine which data storage system to store it in. This allows for timely retrieval of the latest live streaming data when processing is required.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across at least two network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of this application. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A live streaming data processing platform, characterized in that, The live streaming data processing platform includes a streaming media server, a streaming media data acquisition system, a message queue, a first data processing system, a second data processing system, a first data storage system, and a second data storage system, wherein: The streaming media server is used to store the live streaming data of the broadcaster; The streaming media data acquisition system is used to acquire the live streaming data from the streaming media server, and add an acquisition timestamp to the acquired live streaming data after each acquisition. The streaming media data acquisition system is also used to deliver live streaming data with acquisition timestamps to the message queue, and to deliver live streaming data with acquisition timestamps to the first data storage system through a preset delivery method. The first data processing system is used to consume data from the message queue, store the consumed data in the second data storage system, and update the push time of the live stream that matches the first live stream name stored in the second data storage system according to the first live stream name and the collection timestamp contained in the consumed data. The second data processing system is used to consume data delivered through the preset delivery method, store the consumed data in the first data storage system, and update the push time of the live stream that matches the second live stream name stored in the first data storage system according to the second live stream name and the collection timestamp contained in the consumed data.

2. The live streaming data processing platform according to claim 1, characterized in that, The live streaming data processing platform also includes a data aggregation system, which is used for: The push time of each live stream is obtained from the first data storage system and the second data storage system; The latest push time of each live stream is determined based on the push time of each live stream obtained from the first data storage system and the second data storage system.

3. The live streaming data processing platform according to claim 2, characterized in that, The streaming media data acquisition system includes a data acquisition command sending module, a data acquisition module, and a timestamp addition module, wherein: A data acquisition instruction sending module is used to send a data acquisition instruction to the streaming media server. The data acquisition instruction carries a third live stream name of the live stream to be acquired. After receiving the data acquisition instruction, the streaming media server transmits the live streaming data matching the third live stream name to the streaming media data acquisition system. The data acquisition module is used to collect live streaming data transmitted by the streaming media server; The timestamp addition module is used to add a timestamp to the live streaming data after the data acquisition module acquires a single live streaming data.

4. The live streaming data processing platform according to claim 3, characterized in that, The timestamp addition module is further configured to obtain the NTP time after the data acquisition module acquires each live streaming data, and add the NTP time as the timestamp to the acquired live streaming data.

5. The live streaming data processing platform according to claim 1, characterized in that, The preset delivery method is a remote procedure call. The streaming media data acquisition system is also used to deliver live streaming data with acquisition timestamps to the first data storage system via a remote procedure call.

6. The live streaming data processing platform according to any one of claims 1 to 5, characterized in that, Both the first data storage system and the second data storage system are Redis databases.

7. The live streaming data processing platform according to any one of claims 1 to 5, characterized in that, The streaming media server has multiple servers, and the streaming media data acquisition system is also used to collect live streaming data of all broadcasters who are in live streaming status from the multiple streaming media servers.

8. A live streaming data processing method, applied in a live streaming data processing platform, characterized in that, The live streaming data processing platform includes a streaming media server, a streaming media data acquisition system, a message queue, a first data processing system, a second data processing system, a first data storage system, and a second data storage system. The method includes: The streaming media data acquisition system collects the live streaming data of the anchor stored in the streaming media server, and adds a collection timestamp to the collected live streaming data after each collection. The streaming media data acquisition system delivers live streaming data with acquisition timestamps to the message queue, and the streaming media data acquisition system delivers live streaming data with acquisition timestamps to the first data storage system through a preset delivery method; The first data processing system consumes data from the message queue and stores the consumed data in the second data storage system. The first data processing system also updates the push time of the live stream that matches the first live stream name stored in the second data storage system based on the first live stream name and the collection timestamp contained in the consumed data. The second data processing system consumes the data delivered through the preset delivery method and stores the consumed data in the first data storage system. The second data processing system also updates the push time of the live stream that matches the second live stream name stored in the first data storage system based on the second live stream name and the collection timestamp contained in the consumed data.

9. The live streaming data processing method according to claim 8, characterized in that, The live streaming data processing platform also includes a data aggregation system, and the method further includes: The data aggregation system obtains the push time of each live stream from the first data storage system and the second data storage system; The data aggregation system determines the latest push time of each live stream based on the push time of each live stream obtained from the first data storage system and the second data storage system.

10. The live streaming data processing method according to claim 8, characterized in that, The streaming media data acquisition system collects live streaming data of the broadcasters stored in the streaming media server, and adds a collection timestamp to each piece of live streaming data after it is collected, including: The streaming media data acquisition system sends a data acquisition instruction to the streaming media server. The data acquisition instruction carries the third live stream name of the live stream to be acquired. After receiving the data acquisition instruction, the streaming media server transmits the live push data that matches the third live stream name to the streaming media data acquisition system. The streaming media data acquisition system collects live streaming data transmitted by the streaming media server; After each live streaming data acquisition system acquires a live streaming data, it adds a timestamp to the acquired live streaming data.

Images