Live broadcast lagging processing method and device, medium and program product

Abstract

The invention provides a live broadcast lagging processing method, live broadcast lagging processing equipment, a medium and a program product. A specific embodiment of the live broadcast lagging processing method in the application comprises the following steps: receiving a live broadcast transcoding service lagging event; querying live broadcast stream data and transcoding stream data corresponding to the live broadcast transcoding service lagging event; determining live broadcast stream stability and transcoding stream stability based on the live broadcast stream data and the transcoding stream data; and in response to determining that the live broadcast stream is stable but the transcoding stream is unstable, sending a re-transcoding event, the re-transcoding event being used for restarting the transcoding stream. According to the embodiment of the invention, the condition that the stream pushing of the anchor is stable but the production quality of the transcoding stream is abnormal is accurately judged, and the transcoding task is restarted by automatically triggering the transcoding system, so that the transcoding stream is reproduced and recovered to a stable state.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, medium and program product for processing freezes in a live broadcast. Background Art

[0002] Live transcoding refers to converting a compressed and encoded video stream into another video stream to adapt to different network bandwidths, different terminal processing capabilities and different user needs.

[0003] In the live broadcast business scenario system, live broadcast transcoding is a very common means to improve user experience and reduce costs. However, live broadcast transcoding resources and anchor streaming resources are not matched one by one, so there are many transcoding stream productions that require another transcoding resource node to pull the stream and then produce the transcoding stream. However, in this process, the transcoding stream will be affected by many factors, resulting in particularly poor production quality of the transcoding stream when the source stream is not stuck. Summary of the invention

[0004] Multiple aspects of the present application provide a method, device, medium and program product for processing live broadcast freezes, so as to solve the problem of poor production quality of transcoded streams when the source stream is not freezed.

[0005] One aspect of the present application provides a method for processing live broadcast freezes, including: receiving a live broadcast transcoding service freeze event; querying live stream data and transcoded stream data corresponding to the live broadcast transcoding service freeze event; determining the live stream stability and the transcoded stream stability based on the live stream data and the transcoded stream data; in response to determining that the live stream is stable but the transcoded stream is unstable, sending a re-transcoding event, wherein the re-transcoding event is used to restart the transcoded stream.

[0006] Another aspect of the present application provides a live broadcast freeze processing device, including: a receiving module, configured to receive a live broadcast transcoding service freeze event; a query module, configured to query the live broadcast stream data and transcoding stream data corresponding to the live broadcast transcoding service freeze event; a determination module, configured to determine the live broadcast stream stability and the transcoding stream stability based on the live broadcast stream data and the transcoding stream data; a sending module, configured to send a re-transcoding event in response to determining that the live broadcast stream is stable but the transcoding stream is unstable, wherein the re-transcoding event is used to restart the transcoding stream.

[0007] On the other hand, the present application provides a live broadcast freeze processing system, including a freeze data push subsystem, a live broadcast real-time streaming data subsystem, a live broadcast transcoding freeze analysis subsystem and a live broadcast transcoding subsystem; the freeze data push subsystem is used to send the live broadcast transcoding service freeze event to the live broadcast transcoding freeze analysis subsystem; the live broadcast transcoding freeze analysis subsystem is used to query the live broadcast stream data and transcoded stream data corresponding to the live broadcast transcoding service freeze event in the live broadcast real-time streaming data subsystem; based on the live broadcast stream data and the transcoded stream data, determine the live broadcast stream stability and the transcoded stream stability; in response to determining that the live broadcast stream is stable but the transcoded stream is unstable, send a re-transcoding event to the live broadcast transcoding subsystem; the live broadcast transcoding subsystem is used to restart the transcoded stream corresponding to the re-transcoding event.

[0008] Another aspect of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the live broadcast freeze processing method as described above.

[0009] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the live broadcast freeze processing method as described above.

[0010] In another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the live broadcast freeze processing method as described above when executed by a processor.

[0011] The embodiment of the present application provides a method for handling live broadcast freezes, which accurately determines the situation where the host's streaming is stable but the transcoding stream production quality is abnormal by combining multiple dimensions such as barrage card messages, live broadcast room disconnection messages, anchor push quality, and transcoding stream production quality. Then, the transcoding task is restarted by automatically triggering the transcoding system, so that the transcoding stream can be re-produced and restored to a stable state. On the one hand, it can reduce the accidental restart of the transcoding stream due to system misjudgment, and on the other hand, it fundamentally solves the problem of clarity being disconnected due to quality problems. The ability to repair the clarity problem without the user noticing greatly improves the user's viewing experience and reduces bandwidth costs to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0013] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0014] Figure 1 A processing flow chart of an embodiment of a method for processing live broadcast freezes provided by the present application;

[0015] Figure 2 A processing flow chart of another embodiment of the live broadcast freeze processing method provided by the present application;

[0016] Figure 3 A schematic diagram of the structure of an embodiment of a live broadcast freeze processing device provided by the present application;

[0017] Figure 4 An interactive timing diagram of an embodiment of a live broadcast freeze processing system provided by the present application;

[0018] Figure 5 A schematic diagram of the structure of a device suitable for implementing the solution in the embodiment of the present application;

[0019] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.

[0022] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0023] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer program instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0024] An embodiment of the present application provides a method for processing live broadcast freezes. After receiving a live broadcast transcoding service freeze event, the live broadcast stream data and transcoded stream data corresponding to the live broadcast transcoding service freeze event are queried; then, based on the live broadcast stream data and the transcoded stream data, the live broadcast stream stability and the transcoded stream stability are determined; after determining that the live broadcast stream is stable but the transcoded stream is unstable, a re-transcoding event is sent to restart the transcoded stream.

[0025] By combining multiple dimensions such as barrage message, live broadcast room disconnection message, anchor push stream quality and transcoding stream production quality, we can accurately judge the situation where the anchor push stream is stable but the transcoding stream production quality is abnormal. Then, the transcoding task is restarted by automatically triggering the transcoding system, so that the transcoding stream can be re-produced and restored to a stable state. On the one hand, it can reduce the accidental restart of the transcoding stream due to system misjudgment, and on the other hand, it fundamentally solves the problem of clarity disconnection due to quality problems. Being able to repair the problem clarity without the user noticing greatly improves the user's viewing experience and reduces bandwidth costs to a certain extent.

[0026] In actual scenarios, the execution subject of the above method can be a user device, or a network device, or a device formed by integrating a user device and a network device through a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablets, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or computer sets based on cloud computing. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0027] Figure 1The processing flow 100 of an embodiment of a method for processing live broadcast freeze provided by the present application is shown, and the method at least includes the following processing steps:

[0028] Step S101, receiving a live transcoding service freeze event.

[0029] In this embodiment, the execution subject of the live broadcast freeze processing method can receive a live broadcast transcoding service freeze event.

[0030] In the live broadcast architecture, the anchor can push the live broadcast stream to the edge computing node in the cloud. At the same time, the transcoding system can pull the live broadcast stream from the edge computing node and output the transcoded stream with multiple encoding formats and bit rates after processing. The audience can use the player to pull the transcoded stream from the CDN (Content Delivery Network) back to the upstream node to watch, thus forming an overall live broadcast link process.

[0031] When watching the transcoded stream played in the live broadcast room through the player, the audience can send corresponding bullet screen messages according to their viewing experience. Usually, when the transcoded stream is stuck, the audience will send bullet screen messages. In addition, when the player plays the transcoded stream in the live broadcast room, it can detect in real time whether the live broadcast room is stuck, and report the live broadcast room stuck message when it detects that the live broadcast room is stuck. By collecting bullet screen messages and live broadcast room stuck messages in real time and analyzing them, it can be determined whether the live broadcast transcoding service is stuck.

[0032] Through the sliding window, the number of bullet screen call-out messages and live broadcast room disconnection messages in a live broadcast room can be dynamically maintained. If the number of bullet screen call-out messages and live broadcast room disconnection messages simultaneously meets the pre-set jamming judgment configuration, a live broadcast transcoding service jamming event can be triggered. For example, within the preset time window, the number of bullet screen call-out messages collected in real time is greater than the first preset threshold, and the number of live broadcast room disconnection messages reported in real time is greater than the second preset threshold, it can be determined that a live broadcast transcoding service jamming event has occurred.

[0033] Step S102, querying the live streaming data and transcoding streaming data corresponding to the freeze event of the live transcoding service.

[0034] In this embodiment, the above-mentioned execution subject can query the live streaming data and transcoding streaming data corresponding to the freeze event of the live transcoding service.

[0035] The live streaming data may be data related to the live streaming, including but not limited to: the live streaming bitrate, streaming frame rate, streaming node quality, streaming encoding format, etc. Similarly, the transcoded streaming data may be data related to the transcoded streaming, including but not limited to: the transcoded streaming bitrate, streaming frame rate, streaming node quality, streaming encoding format, etc.

[0036] Step S103, determining the stability of the live stream and the stability of the transcoded stream based on the live stream data and the transcoded stream data.

[0037] In this embodiment, the above-mentioned execution subject can determine the live stream stability and the transcoded stream stability based on the live stream data and the transcoded stream data.

[0038] The streaming jitter algorithm is used to calculate the streaming data to determine whether the streaming is stable. The streaming can be a live streaming or a transcoded streaming.

[0039] In some embodiments, first obtain the FPS (Frame PerSecond, frames per second) sequence of the stream within a preset time period before the current moment; then calculate the average value of the FPS sequence; finally compare the average value of the FPS sequence with the preset FPS to determine whether the stream is stable. For example, for the current moment t0, obtain the FPS sequence of the stream within the time period [t0-5min, t0]. Among them, one FPS data corresponds to each second, and the FPS sequence has about 300 FPS data. The FPS data in the FPS sequence is averaged. If the average value is near the preset FPS, it can be determined that the stream is stable, or other indicators can be used to further determine whether the stream is stable; otherwise, it can be directly determined that the stream is unstable. Among them, the preset FPS can be the FPS of a stable stream in actual applications, generally 25, 30, 60, etc.

[0040] In some embodiments, after calculating the average value of the FPS sequence, the difference between the average value of the FPS sequence and the preset FPS can be calculated. If the difference is less than the preset difference threshold, the maximum and minimum values ​​in the FPS sequence can be further compared with the preset FPS to determine whether the flow is stable; otherwise, it can be directly determined that the flow is unstable. For example, if the average value is near the preset FPS, the maximum and minimum values ​​in the FPS sequence can be calculated, and the deviation from the preset FPS. If the deviation is not less than the preset deviation threshold, it can be determined that the flow is unstable, or other indicators can be used to further determine whether the flow is stable; otherwise, it can be directly determined that the flow is stable. Among them, the deviation can represent the degree to which the maximum or minimum value deviates from the preset FPS. The deviation can be obtained by first calculating the absolute value of the difference between the maximum or minimum value and the preset FPS, and then dividing it by the preset FPS.

[0041] In some embodiments, if the deviation is not less than a preset deviation threshold, it indicates that the flow is jittery. At this time, the variance of the FPS sequence can be further calculated, and whether the flow is stable can be determined based on the variance of the FPS sequence. Among them, if the variance of the FPS sequence is less than the preset variance threshold, it indicates that the flow is jittery instantaneously, and the flow can be directly determined to be stable; otherwise, it indicates that the flow is continuously jittering, and the flow can be directly determined to be unstable.

[0042] Step S104: in response to determining that the live stream is stable but the transcoded stream is unstable, a re-transcoding event is sent.

[0043] In this embodiment, when the live stream is stable but the transcoded stream is unstable, the execution subject may send a re-transcoding event.

[0044] The re-transcoding event can be used to restart the transcoding stream, for example, by querying the generation process of the transcoding stream corresponding to the re-transcoding event and re-calling new transcoding resources to generate the transcoding stream, thereby achieving automatic repair of the transcoding stream.

[0045] It should be noted that if the live stream is stable but the transcoding stream is unstable, it means that the host's push stream is stable but the transcoding stream production quality is abnormal. At this time, the transcoding system can be automatically triggered in real time to restart the transcoding task, so that the transcoding stream can be re-produced and restored to a stable state. If the live stream is unstable and the transcoding stream is unstable, it means that the host's push stream is unstable. At this time, restarting the transcoding task cannot improve the transcoding stream production quality. Therefore, it is necessary to switch the edge computing node for live stream push to ensure that the host's push stream is stable first. If the live stream is stable and the transcoding stream production quality is normal, it means that the host's push stream is stable and the transcoding stream production is stable. At this time, the received live transcoding service jam event is a system misjudgment, and there is no need to restart the transcoding task. The transcoding stream will only be restarted when the live stream is stable but the transcoding stream is unstable, which can reduce the accidental restart of the transcoding stream due to system misjudgment.

[0046] The embodiment of the present application provides a method for handling live broadcast freezes, which accurately determines the situation where the host's streaming is stable but the transcoding stream production quality is abnormal by combining multiple dimensions such as barrage card messages, live broadcast room disconnection messages, anchor push quality, and transcoding stream production quality. Then, the transcoding task is restarted by automatically triggering the transcoding system, so that the transcoding stream can be re-produced and restored to a stable state. On the one hand, it can reduce the accidental restart of the transcoding stream due to system misjudgment, and on the other hand, it fundamentally solves the problem of clarity being disconnected due to quality problems. The ability to repair the clarity problem without the user noticing greatly improves the user's viewing experience and reduces bandwidth costs to a certain extent.

[0047] Figure 2 A processing flow 200 of another embodiment of a method for processing live broadcast freeze provided by the present application is shown, and the method at least includes the following processing steps:

[0048] Step S201, receiving a live transcoding service freeze event.

[0049] In this embodiment, the execution subject of the live broadcast freeze processing method can receive a live broadcast transcoding service freeze event.

[0050] In the live broadcast architecture, the anchor can push the live broadcast stream to the edge computing node in the cloud. At the same time, the transcoding system can pull the live broadcast stream from the edge computing node and output the transcoded stream with multiple encoding formats and bit rates after processing. The audience can use the player to pull the transcoded stream from the CDN (Content Delivery Network) back to the upstream node to watch, thus forming an overall live broadcast link process.

[0051] When watching the transcoded stream played in the live broadcast room through the player, the audience can send corresponding bullet screen messages according to their viewing experience. Usually, when the transcoded stream is stuck, the audience will send bullet screen messages. In addition, when the player plays the transcoded stream in the live broadcast room, it can detect in real time whether the live broadcast room is stuck, and report the live broadcast room stuck message when it detects that the live broadcast room is stuck. By collecting bullet screen messages and live broadcast room stuck messages in real time and analyzing them, it can be determined whether the live broadcast transcoding service is stuck.

[0052] Through the sliding window, the number of bullet screen call-out messages and live broadcast room disconnection messages in a live broadcast room can be dynamically maintained. If the number of bullet screen call-out messages and live broadcast room disconnection messages simultaneously meets the pre-set jamming judgment configuration, a live broadcast transcoding service jamming event can be triggered. For example, within the preset time window, the number of bullet screen call-out messages collected in real time is greater than the first preset threshold, and the number of live broadcast room disconnection messages reported in real time is greater than the second preset threshold, it can be determined that a live broadcast transcoding service jamming event has occurred.

[0053] Step S202, query the live streaming data and transcoding streaming data corresponding to the freeze event of the live transcoding service.

[0054] In this embodiment, the above-mentioned execution subject can query the live streaming data and transcoding streaming data corresponding to the freeze event of the live transcoding service.

[0055] The live streaming data may be data related to the live streaming, including but not limited to: the live streaming bitrate, streaming frame rate, streaming node quality, streaming encoding format, etc. Similarly, the transcoded streaming data may be data related to the transcoded streaming, including but not limited to: the transcoded streaming bitrate, streaming frame rate, streaming node quality, streaming encoding format, etc.

[0056] Step S203, obtaining a sequence of frames per second of a stream in a preset time period before the current moment, wherein the stream is a live stream or a transcoded stream.

[0057] In this embodiment, the execution subject may obtain the FPS sequence of a stream within a preset time period before the current moment, wherein the stream is a live stream or a transcoded stream.

[0058] By calculating the streaming data through the push streaming jitter algorithm, it can be determined whether the stream is stable. The stream can be a live stream or a transcoded stream. The streaming data can be the FPS sequence of the stream in a preset time period before the current moment. Each second corresponds to one FPS data, and the FPS sequence has about 300 FPS data.

[0059] Step S204, calculating the average value of the frame number sequence per second.

[0060] In this embodiment, the execution entity may calculate an average value of the FPS sequence.

[0061] Step S205 , determining whether the difference between the average value and the preset number of frames per second is less than a preset difference threshold.

[0062] In this embodiment, the execution subject may compare the difference between the average value of the FPS sequence and the preset FPS with a preset difference threshold. If the difference is less than the preset difference threshold, execute step S206; if the difference is not less than the preset difference threshold, execute step S211.

[0063] Step S206, calculating the maximum and minimum values ​​in the frame rate per second sequence, and the degree of deviation from the preset frame rate per second.

[0064] In this embodiment, if the difference is less than a preset difference threshold, the execution entity may calculate the maximum and minimum values ​​in the FPS sequence and the degree of deviation from the preset FPS.

[0065] If the difference is less than the preset difference threshold, it means that the average value of the FPS sequence is near the preset FPS. The maximum and minimum values ​​in the FPS sequence can be further used to determine whether the stream is stable by the deviation from the preset frames per second. The deviation can indicate the degree to which the maximum or minimum value deviates from the preset FPS. The deviation can be obtained by first calculating the absolute value of the difference between the maximum or minimum value and the preset FPS, and then dividing it by the preset FPS.

[0066] Step S207, determining whether the deviation is less than a preset deviation threshold.

[0067] In this embodiment, the execution entity may compare the deviation with a preset deviation threshold. If the deviation is not less than the preset deviation threshold, step S208 is executed; if the deviation is less than the preset deviation threshold, step S210 is executed.

[0068] Step S208, calculating the variance of the frame number sequence per second.

[0069] In this embodiment, if the deviation is not less than the preset deviation threshold, it indicates that the flow has jitter. At this time, the variance of the FPS sequence can be further calculated to determine whether the flow is stable.

[0070] Step S209: determine whether the variance is less than a preset variance threshold.

[0071] In this embodiment, the execution subject may compare the variance with a preset variance threshold, and if the variance is less than the preset variance threshold, execute step S210; if the variance is not less than the preset variance threshold, execute step S211.

[0072] Step S210, determining whether the flow is stable.

[0073] In this embodiment, if the difference between the average value of the FPS sequence and the preset FPS is less than the preset difference threshold, and the maximum and minimum values ​​in the FPS sequence have a deviation from the preset FPS that is less than the preset deviation threshold, the above-mentioned execution subject can directly determine that the flow is stable. If the difference between the average value of the FPS sequence and the preset FPS is less than the preset difference threshold, the maximum and minimum values ​​in the FPS sequence have a deviation from the preset FPS that is not less than the preset deviation threshold, and the variance of the FPS sequence is less than the preset variance threshold, it indicates that the flow is jittering instantaneously, and the above-mentioned execution subject can directly determine that the flow is stable.

[0074] Step S211, determining that the flow is unstable.

[0075] In this embodiment, if the difference between the average value of the FPS sequence and the preset FPS is not less than the preset difference threshold, it means that the average value of the FPS sequence is not near the preset FPS, and the above-mentioned execution subject can directly determine that the flow is unstable. If the difference between the average value of the FPS sequence and the preset FPS is less than the preset difference threshold, the maximum and minimum values ​​in the FPS sequence have a deviation from the preset FPS that is not less than the preset deviation threshold, and the variance of the FPS sequence is not less than the preset variance threshold, it means that the flow continues to jitter, and the above-mentioned execution subject can directly determine that the flow is unstable.

[0076] Step S212: In response to determining that the live stream is stable but the transcoded stream is unstable, a re-transcoding event is sent.

[0077] The re-transcoding event can be used to restart the transcoding stream, for example, by querying the generation process of the transcoding stream corresponding to the re-transcoding event and re-calling new transcoding resources to generate the transcoding stream, thereby achieving automatic repair of the transcoding stream.

[0078] It should be noted that if the live stream is stable but the transcoding stream is unstable, it means that the host's push stream is stable but the transcoding stream production quality is abnormal. At this time, the transcoding system can be automatically triggered in real time to restart the transcoding task, so that the transcoding stream can be re-produced and restored to a stable state. If the live stream is unstable and the transcoding stream is unstable, it means that the host's push stream is unstable. At this time, restarting the transcoding task cannot improve the transcoding stream production quality. Therefore, it is necessary to switch the edge computing node for live stream push to ensure that the host's push stream is stable first. If the live stream is stable and the transcoding stream production quality is normal, it means that the host's push stream is stable and the transcoding stream production is stable. At this time, the received live transcoding service jam event is a system misjudgment, and there is no need to restart the transcoding task. The transcoding stream will only be restarted when the live stream is stable but the transcoding stream is unstable, which can reduce the accidental restart of the transcoding stream due to system misjudgment.

[0079] The embodiment of the present application provides a method for handling live broadcast freezes, which accurately determines the situation where the host's streaming is stable but the transcoding stream production quality is abnormal by combining multiple dimensions such as barrage card messages, live broadcast room disconnection messages, anchor push quality, and transcoding stream production quality. Then, the transcoding task is restarted by automatically triggering the transcoding system, so that the transcoding stream can be re-produced and restored to a stable state. On the one hand, it can reduce the accidental restart of the transcoding stream due to system misjudgment, and on the other hand, it fundamentally solves the problem of clarity being disconnected due to quality problems. The ability to repair the clarity problem without the user noticing greatly improves the user's viewing experience and reduces bandwidth costs to a certain extent.

[0080] Figure 3 A schematic diagram of the structure of an embodiment of a live broadcast freeze processing device provided by the present application is shown. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0081] like Figure 3 As shown, the live broadcast freeze processing device 300 of this embodiment may include: a receiving module 301, a query module 302, a determination module 303 and a sending module 304. The receiving module 301 is configured to receive a freeze event of a live broadcast transcoding service; the query module 302 is configured to query the live broadcast stream data and the transcoded stream data corresponding to the freeze event of the live broadcast transcoding service; the determination module 303 is configured to determine the stability of the live broadcast stream and the stability of the transcoded stream based on the live broadcast stream data and the transcoded stream data; the sending module 304 is configured to send a re-transcoding event in response to determining that the live broadcast stream is stable but the transcoded stream is unstable, wherein the re-transcoding event is used to restart the transcoded stream.

[0082] In this embodiment, in the live broadcast freeze processing device 300, the specific processing of the receiving module 301, the query module 302, the determination module 303 and the sending module 304 and the technical effects thereof can be referred to in detail. Figure 1 The relevant descriptions of steps S101 to S104 in the corresponding embodiment are not repeated here.

[0083] In some optional implementations of this embodiment, the determination module 303 is further configured to: obtain a frame number sequence per second of a stream within a preset time period before the current moment, wherein the stream is a live stream or a transcoded stream; calculate an average value of the frame number sequence per second; compare the average value of the frame number sequence per second with a preset frame number per second to determine the stream stability.

[0084] In some optional implementations of the present embodiment, the determination module 303 is further configured to: calculate the difference between the average value of the frame number sequence per second and the preset frame number per second; if the difference is not less than the preset difference threshold, determine that the flow is unstable; if the difference is less than the preset difference threshold, compare the maximum and minimum values ​​in the frame number sequence per second with the preset frame number per second to determine the flow stability.

[0085] In some optional implementations of the present embodiment, the determination module 303 is further configured to: calculate the maximum and minimum values ​​in the frame number per second sequence, and the deviation from the preset frame number per second; if the deviation is less than the preset deviation threshold, determine that the flow is stable; if the deviation is not less than the preset deviation threshold, calculate the variance of the frame number per second sequence; based on the variance of the frame number per second sequence, determine the flow stability.

[0086] In some optional implementations of this embodiment, the determination module 303 is further configured to: if the variance of the frame number sequence per second is less than a preset variance threshold, determine that the flow is stable; if the variance of the frame number sequence per second is not less than the preset variance threshold, determine that the flow is unstable.

[0087] In some optional implementations of the present embodiment, within a preset time window, the number of barrage cut messages collected in real time is greater than a first preset threshold, and the number of live broadcast room disconnection messages reported in real time is greater than a second preset threshold, and it is determined that a live broadcast transcoding service freeze event has occurred.

[0088] In some optional implementations of this embodiment, the transcoding stream generation process corresponding to the re-transcoding event is queried, and a new transcoding resource is re-called to generate the transcoding stream.

[0089] Figure 4 The interactive sequence 400 of an embodiment of the live broadcast freeze processing system provided by the present application is shown. The system includes a freeze data push subsystem, a live broadcast real-time streaming data subsystem, a live broadcast transcoding freeze analysis subsystem and a live broadcast transcoding subsystem, and performs at least the following interactive steps:

[0090] Step S401: The jam data push subsystem sends the live transcoding service jam event to the live transcoding jam analysis subsystem.

[0091] In this embodiment, the jamming data push subsystem sends the live transcoding service jamming event to the live transcoding jamming analysis subsystem.

[0092] In the live broadcast architecture, the anchor can push the live broadcast stream to the edge computing node in the cloud. At the same time, the transcoding system can pull the live broadcast stream from the edge computing node and output the transcoded stream with multiple encoding formats and bit rates after processing. The audience can use the player to pull the transcoded stream from the CDN back to the upstream node to watch, thus forming an overall live broadcast link process.

[0093] When the audience is watching the transcoded stream played in the live broadcast room through the player, they can send corresponding bullet screen messages according to their viewing experience. Usually, when the transcoded stream is stuck, the audience will send bullet screen messages. In addition, when the player is playing the transcoded stream in the live broadcast room, it can detect in real time whether the live broadcast room is stuck, and report the live broadcast room stuck message when it detects that the live broadcast room is stuck. The stuck data push subsystem can determine whether there is a stuck event in the live broadcast transcoding service by collecting bullet screen messages and live broadcast room stuck messages in real time and analyzing them.

[0094] Through the sliding window, the number of bullet screen call-out messages and live broadcast room disconnection messages in a live broadcast room can be dynamically maintained. If the number of bullet screen call-out messages and live broadcast room disconnection messages simultaneously meets the pre-set jamming judgment configuration, a live broadcast transcoding service jamming event can be triggered. For example, within the preset time window, the number of bullet screen call-out messages collected in real time is greater than the first preset threshold, and the number of live broadcast room disconnection messages reported in real time is greater than the second preset threshold, it can be determined that a live broadcast transcoding service jamming event has occurred.

[0095] Step S402, the live transcoding freeze analysis subsystem queries the live streaming data subsystem for the live streaming data and the transcoding streaming data corresponding to the freeze event of the live transcoding service.

[0096] In this embodiment, the live transcoding freeze analysis subsystem queries the live streaming data and transcoding streaming data corresponding to the live transcoding service freeze event in the live real-time streaming data subsystem.

[0097] The live streaming real-time push data subsystem can be responsible for maintaining the original push data of all streams. The monitoring mainly includes multiple dimensional data such as the stream push bit rate, stream push frame rate, stream push node quality, stream push encoding format, etc., providing the most original push streaming media data to other business parties.

[0098] The live streaming data may be data related to the live streaming, including but not limited to: the live streaming bitrate, streaming frame rate, streaming node quality, streaming encoding format, etc. Similarly, the transcoded streaming data may be data related to the transcoded streaming, including but not limited to: the transcoded streaming bitrate, streaming frame rate, streaming node quality, streaming encoding format, etc.

[0099] Step S403: the live transcoding jam analysis subsystem determines the live stream stability and the transcoding stream stability based on the live stream data and the transcoding stream data.

[0100] In this embodiment, the live transcoding jamming analysis subsystem determines the live stream stability and the transcoding stream stability based on the live stream data and the transcoding stream data.

[0101] The streaming jitter algorithm is used to calculate the streaming data to determine whether the streaming is stable. The streaming can be a live streaming or a transcoded streaming.

[0102] In some embodiments, first obtain the FPS sequence of the flow in the preset time period before the current moment; then calculate the average value of the FPS sequence; finally compare the average value of the FPS sequence with the preset FPS to determine whether the flow is stable. For example, for the current moment t0, obtain the FPS sequence of the flow in the time period [t0-5min, t0]. Among them, one FPS data corresponds to each second, and the FPS sequence has about 300 FPS data. The FPS data in the FPS sequence is averaged. If the average value is near the preset FPS, it can be determined that the flow is stable, or other indicators can be used to further determine whether the flow is stable; otherwise, it can be directly determined that the flow is unstable. Among them, the preset FPS can be the FPS of a stable flow in actual applications, generally 25, 30, 60, etc.

[0103] In some embodiments, after calculating the average value of the FPS sequence, the difference between the average value of the FPS sequence and the preset FPS can be calculated. If the difference is less than the preset difference threshold, the maximum and minimum values ​​in the FPS sequence can be further compared with the preset FPS to determine whether the flow is stable; otherwise, it can be directly determined that the flow is unstable. For example, if the average value is near the preset FPS, the maximum and minimum values ​​in the FPS sequence can be calculated, and the deviation from the preset FPS. If the deviation is not less than the preset deviation threshold, it can be determined that the flow is unstable, or other indicators can be used to further determine whether the flow is stable; otherwise, it can be directly determined that the flow is stable. Among them, the deviation can represent the degree to which the maximum or minimum value deviates from the preset FPS. The deviation can be obtained by first calculating the absolute value of the difference between the maximum or minimum value and the preset FPS, and then dividing it by the preset FPS.

[0104] In some embodiments, if the deviation is not less than a preset deviation threshold, it indicates that the flow is jittery. At this time, the variance of the FPS sequence can be further calculated, and whether the flow is stable can be determined based on the variance of the FPS sequence. Among them, if the variance of the FPS sequence is less than the preset variance threshold, it indicates that the flow is jittery instantaneously, and the flow can be directly determined to be stable; otherwise, it indicates that the flow is continuously jittering, and the flow can be directly determined to be unstable.

[0105] Step S404: the live transcoding freeze analysis subsystem determines that the live stream is stable but the transcoding stream is unstable.

[0106] In this embodiment, the live transcoding jam analysis subsystem determines that the live stream is stable but the transcoding stream is unstable. Among them, the live stream is stable but the transcoding stream is unstable, which means that the anchor's streaming is stable but the transcoding stream production quality is abnormal.

[0107] Step S405: the live transcoding jam analysis subsystem sends the re-transcoding event to the live transcoding subsystem.

[0108] In this embodiment, the live transcoding jam analysis subsystem sends a re-transcoding event to the live transcoding subsystem, wherein the re-transcoding event can be used to restart the transcoding stream.

[0109] Step S406: the live broadcast transcoding subsystem restarts the transcoding stream corresponding to the re-transcoding event.

[0110] In this embodiment, the live broadcast transcoding subsystem restarts the transcoding stream corresponding to the re-transcoding event.

[0111] The re-transcoding event can be used to restart the transcoding stream, for example, by querying the generation process of the transcoding stream corresponding to the re-transcoding event and re-calling new transcoding resources to generate the transcoding stream, thereby achieving automatic repair of the transcoding stream.

[0112] It should be noted that if the live stream is stable but the transcoding stream is unstable, it means that the host's push stream is stable but the transcoding stream production quality is abnormal. At this time, the transcoding system can be automatically triggered in real time to restart the transcoding task, so that the transcoding stream can be re-produced and restored to a stable state. If the live stream is unstable and the transcoding stream is unstable, it means that the host's push stream is unstable. At this time, restarting the transcoding task cannot improve the transcoding stream production quality. Therefore, it is necessary to switch the edge computing node for live stream push to ensure that the host's push stream is stable first. If the live stream is stable and the transcoding stream production quality is normal, it means that the host's push stream is stable and the transcoding stream production is stable. At this time, the received live transcoding service jam event is a system misjudgment, and there is no need to restart the transcoding task. The transcoding stream will only be restarted when the live stream is stable but the transcoding stream is unstable, which can reduce the accidental restart of the transcoding stream due to system misjudgment.

[0113] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the method corresponding to the electronic device may be the live broadcast freeze processing method in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0114] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or may be an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0115] Figure 5 The structure of a device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown, and the device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, the ROM 502 and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0116] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker, etc.; a storage section 508 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0117] In particular, the methods and / or embodiments in the embodiments of the present application may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, the above functions defined in the method of the present application are executed.

[0118] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application described above.

[0119] Specifically, the present embodiment may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device, or device.

[0120] Computer readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0121] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0122] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0123] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0128] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0130] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for processing live broadcast freeze, comprising: Receive the freeze event of the live transcoding service; Query the live streaming data and transcoding streaming data corresponding to the live transcoding service freeze event; Based on the live streaming data and the transcoded streaming data, determining the live streaming stability and the transcoded streaming stability; In response to determining that the live stream is stable but the transcoded stream is unstable, a re-transcoding event is sent, wherein the re-transcoding event is used to restart the transcoded stream.

2. The method according to claim 1, wherein: The determining of the live stream stability and the transcoded stream stability based on the live stream data and the transcoded stream data includes: Obtain a sequence of frames per second of a stream within a preset time period before a current moment, wherein the stream is a live stream or a transcoded stream; Calculating an average value of the sequence of frames per second; The average value of the sequence of frames per second is compared with a preset frame per second to determine the flow stability.

3. The method according to claim 2, wherein: The step of comparing the average value of the frame number per second sequence with a preset frame number per second to determine the flow stability comprises: Calculating the difference between the average value of the frame rate per second sequence and the preset frame rate per second; If the difference is not less than a preset difference threshold, determining that the flow is unstable; If the difference is less than a preset difference threshold, the maximum value and the minimum value in the frame rate sequence per second are compared with the preset frame rate per second to determine the flow stability.

4. The method according to claim 3, wherein: The step of comparing the maximum value and the minimum value in the frame number per second sequence with the preset frame number per second to determine the flow stability includes: Calculating the maximum and minimum values ​​in the frame rate sequence per second, and the degree of deviation from the preset frame rate per second; If the deviation is less than a preset deviation threshold, determining that the flow is stable; If the deviation is not less than a preset deviation threshold, calculating the variance of the frames per second sequence; Based on the variance of the frames per second sequence, flow stability is determined.

5. The method according to claim 4, wherein: The determining of flow stability based on the variance of the sequence of frames per second comprises: If the variance of the frame number sequence per second is less than a preset variance threshold, determining that the flow is stable; If the variance of the frame number sequence per second is not less than a preset variance threshold, it is determined that the flow is unstable.

6. The method according to any one of claims 1 to 5, wherein: Within the preset time window, the number of barrage cut messages collected in real time is greater than the first preset threshold, and the number of live broadcast room disconnection messages reported in real time is greater than the second preset threshold, determining that the live broadcast transcoding service freeze event has occurred.

7. The method according to claim 6, wherein: The transcoding stream generation process corresponding to the re-transcoding event is queried, and a new transcoding resource is re-called to generate the transcoding stream.

8. A live broadcast freeze processing device, comprising: A receiving module, configured to receive a freeze event of a live transcoding service; A query module is configured to query the live streaming data and transcoding streaming data corresponding to the live transcoding service freeze event; A determination module, configured to determine the stability of the live stream and the stability of the transcoded stream based on the live stream data and the transcoded stream data; The sending module is configured to send a re-transcoding event in response to determining that the live stream is stable but the transcoding stream is unstable, wherein the re-transcoding event is used to restart the transcoding stream.

9. A live broadcast freeze processing system, comprising a freeze data push subsystem, a live broadcast real-time streaming data subsystem, a live broadcast transcoding freeze analysis subsystem and a live broadcast transcoding subsystem; The jam data push subsystem is used to send the jam event of the live transcoding service to the live transcoding jam analysis subsystem; The live transcoding jamming analysis subsystem is used to query the live streaming data and transcoding streaming data corresponding to the live transcoding service jamming event in the live real-time streaming data subsystem; Based on the live streaming data and the transcoded streaming data, determining the live streaming stability and the transcoded streaming stability; In response to determining that the live stream is stable but the transcoded stream is unstable, sending a re-transcoding event to the live transcoding subsystem; The live transcoding subsystem is used to restart the transcoding stream corresponding to the re-transcoding event.

10. The system according to claim 9, wherein: The Caton data push subsystem is further used for: Collect the bullet screen and call-off messages of the corresponding live broadcast room of the live stream in real time; Receive live broadcast room disconnection messages reported by players in real time; In response to determining that the number of barrage cut messages within a preset time window is greater than a first preset threshold, and the number of live broadcast room disconnection messages is greater than a second preset threshold, it is determined that the live broadcast transcoding service freeze event has occurred.

11. The system according to claim 9, wherein: The live broadcast transcoding subsystem is further used for: The generation process of the transcoding stream corresponding to the re-transcoding event is queried, and a new transcoding resource is re-called to generate the transcoding stream.

12. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

13. A computer readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 7.

14. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images