Method and apparatus for live streaming control, device, and storage medium

Abstract

Embodiments of the present disclosure provide a method and apparatus for live streaming control, a device, and a storage medium. The method comprises: during live streaming, controlling a content acquisition device to acquire first content at a first acquisition frame rate; on the basis of the acquired first content, generating first live streaming content to be pushed to a client associated with the live streaming; on the basis of a push parameter of the first live streaming content, controlling the frame rate of the content acquisition device to be updated from the first acquisition frame rate to a second acquisition frame rate; and controlling the content acquisition device to acquire second content at the second acquisition frame rate for generating second live streaming content.

Description

Methods, apparatus, devices, and storage media for live streaming control

[0001] This application claims priority to Chinese Patent Application No. 202411514922.4, filed on October 28, 2024, entitled "Method, Apparatus, Device and Storage Medium for Live Broadcast Control", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for live broadcast control. Background Technology

[0003] With the development of computer technology, the internet has become an important platform for information exchange. For example, people can use live streaming to transmit video content in real time, allowing users to watch the video content instantly and interact with other users in real time. During live streaming, people not only expect high-quality and smooth video content, but also expect reduced system resource consumption. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for live streaming control is provided. The method includes: during a live stream, controlling a content acquisition device to acquire first content at a first acquisition frame rate; generating first live stream content based on the acquired first content for pushing to a client associated with the live stream; updating the frame rate of the content acquisition device from the first acquisition frame rate to a second acquisition frame rate based on push parameters of the first live stream content; and controlling the content acquisition device to acquire second content at the second acquisition frame rate for generating second live stream content.

[0005] In a second aspect of this disclosure, an apparatus for live streaming control is provided. The apparatus includes: a first acquisition module configured to control a content acquisition device to acquire first content at a first acquisition frame rate during a live stream; a generation module configured to generate first live stream content based on the acquired first content for pushing to a client associated with the live stream; an update module configured to control the frame rate of the content acquisition device to update from the first acquisition frame rate to a second acquisition frame rate based on push parameters of the first live stream content; and a second acquisition module configured to control the content acquisition device to acquire second content at the second acquisition frame rate for generating second live stream content.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided, including computer-executable instructions, wherein when executed by a processor, the computer-executable instructions implement the method according to a first aspect of this disclosure.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;

[0012] Figure 2 shows a flowchart of a process for live broadcast control according to some embodiments of the present disclosure;

[0013] Figure 3 illustrates a schematic diagram of an example architecture for live streaming control according to some embodiments of the present disclosure;

[0014] Figure 4 shows a schematic structural block diagram of an example device for live broadcast control according to some embodiments of the present disclosure; and

[0015] Figure 5 shows a block diagram of an electronic device capable of implementing several embodiments of the present disclosure. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0018] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0019] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0021] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0022] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0024] As mentioned above, live streaming allows for real-time video content transmission, enabling users to watch and interact with the content instantly. During live streaming, cameras typically capture video. Afterward, the video frames are processed, including cropping, applying effects, and encoding, before being pushed to clients related to the live stream (e.g., the clients of guests or viewers). Traditionally, cameras capture video at a set frame rate. However, the processing speed and push speed of video frames are easily affected by various factors such as the load on the broadcaster's terminal device and network conditions, resulting in dynamic changes. If the camera captures video frames at a frame rate far exceeding the push frame rate, some captured frames will be discarded, causing redundant power consumption for the camera. If the camera captures video frames at a frame rate that is too low, it will affect the quality of the live stream content.

[0025] In view of this, embodiments of this disclosure propose an improved scheme for live streaming control. According to this improved scheme, during live streaming, a content acquisition device is controlled to acquire first content at a first acquisition frame rate. Based on the acquired first content, first live streaming content is generated for pushing to clients associated with the live stream. Based on the push parameters of the first live streaming content, the frame rate of the content acquisition device is updated from the first acquisition frame rate to a second acquisition frame rate. The content acquisition device is then controlled to acquire second content at the second acquisition frame rate for generating second live streaming content.

[0026] In this way, embodiments of the present disclosure can match the capture frame rate of the content acquisition device (e.g., a camera) with the push of live content, thereby maintaining the quality of the live content and avoiding redundant power consumption of the content acquisition device.

[0027] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0028] Example Environment

[0029] Figure 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In environment 100, one or more users 110-1, 110-2, 110-3, ..., 110-N can watch live streams and participate in interactive live stream sessions through their respective associated terminal devices 120-1, 120-2, ..., 120-N. User 110 may also be referred to as a participant, for example. For ease of discussion, users 110-1, 110-2, ..., 110-N may be collectively referred to as user 110 or individually, and terminal devices 120-1, 120-2, ..., 120-N may be collectively referred to as terminal device 120 or individually. Among all users 110, one or more users, such as user 110-1, can establish a live stream session and initiate a live stream through their associated terminal device 120-1.

[0030] In some scenarios, the user 110-1 who initiates the live stream can also be referred to as the host of the live stream session, a host participant, the live streamer, or the live stream administrator. One or more other users 110 may include guest participants 110-N, representing users who participate in the live stream interaction initiated by the host (e.g., live chat). In addition to guest participants in the live stream interaction, other users 110 may also include viewers, listeners, or viewers of the live stream session.

[0031] In some embodiments, the terminal device 120 may have an application installed that can provide live streaming services, or may have access to a website that can provide live streaming services. The user 110 can operate the terminal device 120 to access the corresponding application or website. Accordingly, the terminal device 120 may present a corresponding live streaming interface, which may provide live streaming content, such as audio live streaming content or video live streaming content.

[0032] In some embodiments, the terminal device 120 can also communicate with the server device 130 via the network 132 to provide live streaming services. The server device 130 can also provide functions such as application or website management, configuration, and maintenance. In some embodiments, the server device 130 can store data generated during the live streaming session, including live interaction data generated during live streaming interactions.

[0033] Terminal device 120 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 120 may also support any type of user-facing interface (such as "wearable" circuitry). Server device 130 can be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0034] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0035] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0036] Example process

[0037] Figure 2 illustrates a flowchart of a live broadcast control process 200 according to some embodiments of the present disclosure. For ease of discussion, some embodiments of the present disclosure will be described below in conjunction with the environment 100 in Figure 1 and from the perspective of a terminal device 120, but this is merely exemplary. In some embodiments, the actions described relative to the terminal device may be performed by the terminal device in coordination with the server device.

[0038] In block 210 of process 200, during the live broadcast, terminal device 120 controls a content acquisition device to acquire first content at a first acquisition frame rate. This terminal device 120 can be the terminal device of the broadcaster or guest, or it can be a terminal device providing the live broadcast feed. The first content can include, but is not limited to, video, audio, text, etc. Correspondingly, the content acquisition device can include an image acquisition device (e.g., a camera), an audio acquisition device (e.g., a microphone), or a text acquisition device, etc. As an example, terminal device 120 can be the broadcaster's terminal device. Terminal device 120 can control the camera to acquire the broadcaster's video content. As another example, terminal device 120 can run a target application (e.g., game software), and terminal device 120 can control, for example, an interface acquisition device to acquire the content presented by the target application (e.g., game content).

[0039] The first capture frame rate here can be understood as the frame rate before the content capture device performs a frame rate update. This first capture frame rate can be a fixed value or a range of frame rates. In some embodiments, the terminal device 120 can set the frame rate of the content capture device to the first capture frame rate based on the configuration information of the content capture device. Then, it controls the content capture device to capture the first content at the first capture frame rate. For example, in the initial stage of live streaming, the terminal device 120 can set the camera's frame rate to a predetermined frame rate (e.g., capturing 30 frames per second) according to the camera's configuration information. Then, it controls the camera to capture video at the predetermined frame rate. Of course, the first capture frame rate is not limited to the predetermined frame rate; it can also be the frame rate after one or more frame rate updates to the content capture device.

[0040] In block 220 of process 200, terminal device 120 generates live content (sometimes referred to herein as first live content) based on the acquired first content for push to clients associated with the live stream. Live content may include audio streams, video streams, audio-visual streams, etc. Clients associated with the live stream may include the broadcaster's client, the guest's client, or the audience's client. As an example, terminal device 120 may be the guest's terminal device. Terminal device 120 controls the video and audio content captured by the camera and microphone to generate an audio-visual stream containing the guest's image and sound. Then, terminal device 120 can push this audio-visual stream containing the guest's image and sound to the broadcaster's client via, for example, server device 130. The broadcaster's client can merge the guest's audio-visual stream with the broadcaster's audio-visual stream to form the final live content. As another example, terminal device 120 may be the broadcaster's terminal device. Terminal device 120 can control the video and audio content captured by the camera and microphone to generate an audio-visual stream containing the broadcaster's image and sound. Afterwards, terminal device 120 can push audio and video streams to the guest's or audience's client.

[0041] In some embodiments, the terminal device 120 can perform special effects processing, merging processing, encoding processing, etc., on the first content (e.g., audio, video, audio-visual), to generate live content such as encoded audio streams, video streams, or audio-visual streams. To illustrate the live content generation process, the following will provide an exemplary explanation of the live content generation process in conjunction with FIG3. FIG3 shows a schematic diagram of an example architecture 300 for live streaming control according to some embodiments of the present disclosure. As shown in FIG3, the example architecture 300 shows a camera 310, a storage unit 320, a service function component 330, and a live streaming engine 340, all of which are associated with the terminal device 120 for live streaming.

[0042] After the live stream begins, the terminal device 120 can control the camera 310 to capture video frames at a first capture frame rate, and the captured video frames can be stored in the storage unit 320. If a request to output a frame is received, the latest video frame (i.e., the video frame closest to the current time) stored in the storage unit 320 can be provided to the service function component 330. Here, the frame rate at which the storage unit 320 provides video frames to the service function component 330 can be called the input frame rate. It is understandable that in practical applications, the camera 310 will continuously capture video frames at the first capture frame rate, and the storage unit 320 needs to wait for a request to output a frame before providing the latest video frame to the service function component 330. It is possible that between two adjacent requests to output a frame, the storage unit 320 may receive multiple video frames from the camera 310, but the storage unit 320 only needs to provide the latest video frame to the service function component 330, and the remaining video frames may be discarded. Therefore, in practical applications, the capture frame rate of the camera 310 is usually greater than or equal to the input frame rate.

[0043] In response to receiving the latest video frame, the service function component 330 can determine in box 331 whether the current time has reached the frame output time. If it is determined in box 331 that the current time has reached the frame output time, the service function component 330 can provide the latest video frame to the live streaming engine 340 and update the frame output time (NextFrameTime) of the next video frame. For example, X seconds after the current time can be determined as the frame output time of the next video frame. The service function component 330 can also send a frame output request instruction to the storage unit 320 to request the storage unit 320 to provide video frames again. If it is determined in box 331 that the current time has not reached the frame output time, the determination process in box 331 can be executed again after a predetermined time (e.g., Y seconds).

[0044] In box 341, terminal device 120 can perform special effects processing on video frames through live streaming engine 340. Of course, before or after special effects processing, video frames can also be processed by live streaming engine 340, such as cropping or merging. Special effects processing can be an optional operation and can be omitted in some cases.

[0045] In frame 342, terminal device 120 can perform encoding processing on the video frames after special effects processing through the encoder in live streaming engine 340 to form an encoded video stream or audio-visual stream.

[0046] In box 343, terminal device 120 can push the encoded video stream or audio / video stream to, for example, server device 130 via live streaming engine 340, and server device 130 can then push the encoded video stream or audio / video stream to the client of the guest or the client of the audience. The frame rate at which live streaming engine 340 pushes live content to clients related to the live stream can be called the push frame rate.

[0047] In box 344, terminal device 120 can also perform image rendering on the video frames after special effects processing through live streaming engine 340 to form a display image. Then, this display image is presented through the live streaming interface at terminal device 120. The frame rate at which live streaming engine 340 performs image rendering on video frames can be called the rendering frame rate.

[0048] It should be noted that the above process for generating live content is merely exemplary. In practical applications, any appropriate process can be selected to generate live content for live streaming according to actual needs, and the embodiments disclosed herein do not impose specific limitations on this.

[0049] In the return process 200, within box 230, the terminal device 120, based on the push parameters of the live content, controls the frame rate of the content acquisition device to update from a first acquisition frame rate to a second acquisition frame rate. Specifically, the terminal device 120 can determine the push parameters of the live content. Then, based on the determined push parameters, it can control the frame rate of the content acquisition device to update from the first acquisition frame rate to the second acquisition frame rate. The second acquisition frame rate can be understood as the updated frame rate of the content acquisition device. Updating the frame rate of the content acquisition device can include increasing or decreasing the frame rate of the content acquisition device.

[0050] Regarding the acquisition of push parameters by the terminal device 120, in some embodiments, the terminal device 120 may, in response to receiving push parameters of live content from the server device 130, control the frame rate of the content acquisition device to be updated from a first acquisition frame rate to a second acquisition frame rate based on the received push parameters. As an example, the server device 130 may be configured to send push parameters of live content to the terminal device 120 at predetermined time intervals.

[0051] In other embodiments, the terminal device 120 may determine the push parameters of the live broadcast content in response to the live broadcast meeting preset conditions. Based on the push parameters, the frame rate of the content acquisition device is controlled to be updated from a first acquisition frame rate to a second acquisition frame rate. The preset conditions may include time conditions, device performance conditions related to the live broadcast, etc. As an example, the preset conditions may include time conditions, such as the time interval for determining the push parameters. Specifically, during the live broadcast, the terminal device 120 may determine the time interval between the current time and the time of the previous determination of the push parameters. Subsequently, the terminal device 120 may determine whether the time interval exceeds a preset time threshold. If the time interval since the previous determination of the push parameters exceeds the time threshold, the terminal device 120 may re-determine the push parameters of the live broadcast content.

[0052] For example, if the time interval since the last determination of push parameters exceeds a time threshold, terminal device 120 can send a query request to server device 130 to query the push parameters of the live content. Alternatively, server device 130 can also be configured to send push parameters to terminal device 120 in real time. If the time interval since the last determination of push parameters exceeds the time threshold, terminal device 120 can query the push parameters locally. It should be noted that terminal device 120 is not limited to obtaining push parameters from server device 130; terminal device 120 can also detect the push parameters of the live content itself.

[0053] As another example, the predetermined condition may also indicate that the terminal device 120 has redundant processing capacity for the live content or that the terminal device 120 has insufficient processing capacity for the live content. For example, the predetermined condition may include at least one of the following: conditions for the processor, conditions for memory, conditions for storage devices, conditions for communication networks, etc. The terminal device 120 may determine the push parameters for the live content in response to its own device performance parameters (such as processor utilization, operating frequency, temperature, memory utilization, communication network latency, etc.) meeting the predetermined condition.

[0054] The push parameters here can directly or indirectly indicate the push speed of the live content by the terminal device 120. In some embodiments, the push parameters may include a push frame rate that can directly indicate the push speed of the live content. Alternatively, the push parameters may also include parameters that can indicate the processing capability of the terminal device 120 for the live content, which can indirectly indicate the push speed of the live content. For example, the push parameters may also include encoding parameters that can indicate the encoder's encoding capability for the first content. The encoding parameters may include, for example, the encoder's waiting time, which may be the time required for the encoder to wait for the next video frame to be input after completing the encoding processing of the previous video frame. The longer the waiting time, the greater the encoder's redundant encoding capability for the first content; the shorter the waiting time, the smaller the encoder's redundant encoding capability for the first content. If the waiting time is zero, it indicates that the encoder has no redundant encoding capability for the first content.

[0055] In some embodiments, the push parameters may include encoder encoding parameters. The encoder is used to encode the first content to obtain live content. The terminal device 120 may determine, based on the encoding parameters, whether the encoder has redundant encoding capability for the first content acquired at a first acquisition frame rate. If it is determined that the encoder has redundant encoding capability for the first content, the terminal device 120 may determine that the push parameters satisfy a frame rate increase condition for the content acquisition device (sometimes referred to herein as a second update condition), and the terminal device 120 may increase the frame rate of the content acquisition device from the first acquisition frame rate to the second acquisition frame rate. The encoder encoding parameters may include, but are not limited to, encoder wait time, encoder bitrate, encoder encoding frame rate, etc.

[0056] As an example, the encoding parameters may include the encoder's waiting time, and the terminal device 120 can determine whether the encoder's waiting time is greater than zero. If the encoder's waiting time is greater than zero, it indicates that after the encoder completes the encoding processing of the previous video frame, it needs to wait for the next video frame to be input to the encoder. In this case, it can be determined that the encoder has redundant encoding capability for the first content, and thus it can be determined that the push parameters meet the frame rate increase condition of the content acquisition device. Multiple predetermined acquisition frame rates can be pre-configured for the content acquisition device, and the first acquisition frame rate can be one of the multiple predetermined acquisition frame rates. When it is determined that the push parameters meet the frame rate increase condition, the terminal device 120 can determine a second acquisition frame rate that is above the first acquisition frame rate among the multiple predetermined acquisition frame rates. For example, multiple predetermined acquisition frame rates (N11, N12), (N21, N22), (N31, N32), etc., can be pre-configured for the camera. The current frame rate of the camera can be (N11, N12). Terminal device 120 can determine a second acquisition frame rate as a predetermined acquisition frame rate that is above and adjacent to the predetermined acquisition frame rates (N11, N12). Then, terminal device 120 can set the camera's frame rate to (N21, N22). In this way, with the terminal device 120 having redundant encoding capabilities, the quality of the live broadcast content can be improved by increasing the acquisition frame rate of the content acquisition device.

[0057] It should be noted that the process of determining the second acquisition frame rate described above is merely exemplary. When the push parameters meet the conditions for increasing the frame rate, the second acquisition frame rate can also be determined through any other appropriate method. For example, the terminal device 120 can determine an indicator of the encoder's redundancy encoding capability for the first content based on the value of the waiting time. The terminal device 120 can then determine the increase in the frame rate of the content acquisition device based on the determined indicator. Subsequently, the second acquisition frame rate can be determined based on the first acquisition frame rate and this increase.

[0058] In some embodiments, the push parameters may further include the push frame rate for pushing live content. Multiple predetermined acquisition frame rates can be matched to multiple push frame rate ranges for pushing live content. A first acquisition frame rate can be matched to a first push frame rate range among the multiple push frame rate ranges. If it is determined based on encoding parameters that the encoder does not have redundant encoding capability for the first content, the terminal device 120 can determine whether the push frame rate is lower than the lower limit of the first push frame rate range. If it is determined that the push frame rate is lower than the lower limit of the first push frame rate range, it can be determined that the push parameters meet the frame rate reduction condition for the content acquisition device (sometimes referred to herein as the first update condition), and the terminal device 120 can control the frame rate of the content acquisition device to decrease from the first acquisition frame rate to the second acquisition frame rate. In this way, the acquisition frame rate of the content acquisition device can be matched with the push frame rate of the live content, which helps to reduce the redundant power consumption of the content acquisition device.

[0059] In some embodiments, the push parameters may also include only the push frame rate for pushing live content. The terminal device 120 can directly determine whether the push frame rate is lower than a first push frame rate lower limit. If the push frame rate is lower than the first push frame rate lower limit, the terminal device 120 can determine that the push parameters meet the condition for reducing the frame rate of the content acquisition device. Furthermore, the terminal device 120 can control the frame rate of the content acquisition device to decrease from a first acquisition frame rate to a second acquisition frame rate. Of course, the above push parameters are merely exemplary. In practical applications, any appropriate parameter that can indicate the processing capability of the terminal device 120 for live content can be selected according to actual needs. The embodiments of this disclosure do not impose specific limitations on this.

[0060] Regarding controlling the frame rate of the content acquisition device to decrease from a first acquisition frame rate to a second acquisition frame rate, in some embodiments, if it is determined that the push parameters meet the frame rate reduction conditions for the content acquisition device, the terminal device 120 can determine whether the push frame rate falls within a range of multiple push frame rate ranges. If it is determined that the push frame rate falls within the second push frame rate range of multiple push frame rate ranges, the terminal device 120 can determine a second sampling frame rate that matches the second push frame rate range among multiple predetermined acquisition frame rates, and then reduce the frame rate of the content acquisition device to the second acquisition frame rate.

[0061] As an example, multiple predetermined acquisition frame rates (N11, N12), (N21, N22), and (N31, N32) of the camera can be matched to multiple push frame rate ranges (M11, M12), (M21, M22), and (M31, M32). Assuming the current acquisition frame rate of the camera is (N31, N32), the terminal device 120 can determine whether the push frame rate is lower than the lower limit M31 of the push frame rate range (M31, M32). If the push frame rate is determined to be lower than M31, the terminal device 120 can determine that the push parameters meet the frame rate reduction condition for the camera, and the terminal device 120 can determine whether the push frame rate falls within one of the multiple push frame rate ranges (M11, M12) and (M21, M22). If the push frame rate falls within the push frame rate range (M21, M22), the terminal device 120 can control the camera's frame rate to decrease to the predetermined acquisition frame rate (N21, N22).

[0062] It is understood that the process of reducing the frame rate of the content acquisition device from the first acquisition frame rate to the second acquisition frame rate described above is merely exemplary. In practical applications, if it is determined that the push parameters meet the conditions for reducing the frame rate, the second acquisition frame rate can also be determined by any other appropriate method. For example, the terminal device 120 can determine the range of changes in the push frame rate within a predetermined event range prior to the current time. The terminal device 120 can determine a second acquisition frame rate that is greater than or equal to the upper limit of this range and less than the first acquisition frame rate.

[0063] In block 240 of process 200, terminal device 120 controls content acquisition device to acquire second content at a second acquisition frame rate for generating further live content (sometimes referred to herein as second live content). As an example, as shown in Figure 3, terminal device 120 can set the frame rate of camera 310 to the second acquisition frame rate. Then, terminal device 120 can control camera 310 to acquire video frames at the second acquisition frame rate. It should be noted that the main difference between the second content and the first content mentioned above is the frame rate. The process of generating further live content (i.e., second live content) based on the second content is the same as or substantially the same as the process of generating live content (i.e., first live content) based on the first content mentioned above. Therefore, the process of generating further live content based on the second content will not be described in detail here; please refer to the explanation of block 220 in process 200.

[0064] Understandably, process 200 can be repeated during the live stream to dynamically adjust the frame rate of the content acquisition device, ensuring that its frame rate consistently matches the live content feed. Within a given time period, the content acquisition device collects content at a set frame rate (the first frame rate) to generate the live content for that period. If the frame rate of the content acquisition device is controlled to update, then in the next time period, the device will collect content at the updated frame rate (the second frame rate) to generate the live content for that period. Based on this, the first and second live content can be understood as part of the same live stream. The first live content can be generated before the frame rate update, based on the first content collected at the previous frame rate. The second live content can be generated after the frame rate update, based on the second content collected at the updated frame rate.

[0065] In this way, embodiments of the present disclosure can match the capture frame rate of the content acquisition device (e.g., a camera) with the push of live content, thereby maintaining the quality of the live content and avoiding redundant power consumption of the content acquisition device.

[0066] Example devices and equipment

[0067] Embodiments of this disclosure also provide corresponding apparatus for implementing the methods or processes described above. Figure 4 shows a schematic structural block diagram of an example apparatus 400 for live broadcast control according to certain embodiments of this disclosure. Apparatus 400 may be implemented as or included in terminal device 120. The various modules / components in apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0068] As shown in Figure 4, the device 400 includes: a first acquisition module 410, a generation module 420, an update module 430, and a second acquisition module 440. The first acquisition module 410 is configured to control a content acquisition device to acquire first content at a first acquisition frame rate during live streaming. The generation module 420 is configured to generate first live streaming content based on the acquired first content for push to clients associated with the live stream. The update module 430 is configured to control the frame rate of the content acquisition device to update from the first acquisition frame rate to a second acquisition frame rate based on the push parameters of the first live streaming content. The second acquisition module 440 is configured to control the content acquisition device to acquire second content at the second acquisition frame rate for generating second live streaming content.

[0069] In some embodiments, the update module 430 is further configured to: determine the push parameters of the first live content in response to the live stream meeting preset conditions; and control the frame rate of the content acquisition device to be updated from the first acquisition frame rate to the second acquisition frame rate based on the push parameters.

[0070] In some embodiments, the preset conditions include the time interval for determining push parameters, and the update module 430 is further configured to: determine the push parameters of the first live content in response to the time interval between the previous determination of push parameters and the live broadcast exceeding a time threshold.

[0071] In some embodiments, the push parameters include a push frame rate for pushing the first live content, the first acquisition frame rate is matched to a first push frame rate range for pushing the first live content, and the update module 430 is further configured to: determine that the push parameters meet a first update condition for the frame rate of the content acquisition device based on determining that the push frame rate is lower than the lower limit of the first push frame rate range; and in response to determining that the push parameters meet the first update condition, control the frame rate of the content acquisition device to decrease from the first acquisition frame rate to the second acquisition frame rate.

[0072] In some embodiments, the push parameters also include encoder encoding parameters, the encoder being used to encode the first content to obtain the first live content, and the update module 430 is further configured to: determine, based on the encoding parameters, whether the encoder has redundant encoding capability for the first content acquired at a first acquisition frame rate; and further determine, based on the determination that the encoder does not have redundant encoding capability for the first content, that the push parameters meet the first update condition.

[0073] In some embodiments, the content acquisition device has multiple predetermined acquisition frame rates, which are respectively matched to multiple push frame rate ranges. The update module 430 is further configured to: in response to determining that the push parameters meet the first update condition, determine whether the push frame rate falls within the push frame rate range of the multiple push frame rate ranges; and based on determining that the push frame rate falls within the second push frame rate range of the multiple push frame rate ranges, control the frame rate of the content acquisition device to decrease from the first acquisition frame rate to the second acquisition frame rate of the multiple predetermined acquisition frame rates that matches the second push frame rate range.

[0074] In some embodiments, the push parameters include encoding parameters of the encoder, which is used to encode the first content to obtain the first live content. The update module 430 is further configured to: determine, based on the encoding parameters, whether the encoder has redundant encoding capability for the first content acquired at a first acquisition frame rate; determine, based on the determination that the encoder has redundant encoding capability for the first content, that the push parameters meet a second update condition for the frame rate of the content acquisition device; and, in response to the push parameters meeting the second update condition, control the frame rate of the content acquisition device to increase from the first acquisition frame rate to the second acquisition frame rate.

[0075] In some embodiments, the first acquisition module 410 is further configured to: set the frame rate of the content acquisition device to a first acquisition frame rate based on the configuration information of the content acquisition device; and control the content acquisition device to acquire the first content at the first acquisition frame rate.

[0076] The units and / or modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 500 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0077] Figure 5 shows a block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 500 shown in Figure 5 is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. The electronic device 500 shown in Figure 5 may include or be implemented as the terminal device 120 of Figure 1, or the device 400 of Figure 4.

[0078] As shown in Figure 5, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processor 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0079] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0080] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 comprising computer-executable instructions configured to perform various methods or actions of various embodiments of the present disclosure.

[0081] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0082] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0083] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0084] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions.

[0085] These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-executable instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0086] Computer-executable instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for live streaming control, comprising: During the live stream, the content capture device is controlled to capture the first content at the first capture frame rate; Based on the collected first content, first live content is generated for push to the client associated with the live stream; Based on the push parameters of the first live content, the frame rate of the content acquisition device is controlled to be updated from the first acquisition frame rate to the second acquisition frame rate. as well as The content acquisition device is controlled to acquire second content at the second acquisition frame rate for use in generating second live content.

2. The method according to claim 1, wherein controlling the frame rate of the content acquisition device to update from the first acquisition frame rate to the second acquisition frame rate based on the push parameters of the first live content includes: In response to the live stream meeting preset conditions, the push parameters of the first live stream content are determined; as well as Based on the push parameters, the frame rate of the content acquisition device is controlled to be updated from the first acquisition frame rate to the second acquisition frame rate.

3. The method according to claim 2, wherein the preset condition includes a time interval for determining the push parameters, and wherein determining the push parameters of the first live stream content in response to the live stream satisfying the preset condition includes: In response to the time interval between the last determination of push parameters and the start of the live broadcast exceeding a time threshold, push parameters for the first live broadcast content are determined.

4. The method of claim 1, wherein the push parameters include a push frame rate for the push of the first live content, the first acquisition frame rate is matched to a first push frame rate range for the push of the first live content, and wherein controlling the frame rate of the content acquisition device to update from the first acquisition frame rate to a second acquisition frame rate based on the push parameters of the first live content includes: Based on the determination that the push frame rate is lower than the lower limit of the first push frame rate range, it is determined that the push parameters meet the first update condition for the frame rate of the content acquisition device; as well as In response to determining that the push parameters meet the first update condition, the frame rate of the content acquisition device is controlled to decrease from the first acquisition frame rate to the second acquisition frame rate.

5. The method of claim 4, wherein the push parameters further include encoding parameters of an encoder, the encoder being used to encode the first content to obtain the first live content, and wherein determining that the push parameters satisfy a first update condition for the frame rate of the content acquisition device includes: Based on the encoding parameters, it is determined whether the encoder has redundant encoding capability for the first content acquired at the first acquisition frame rate; as well as Furthermore, based on the determination that the encoder does not have redundant encoding capability for the first content, it is determined that the push parameters satisfy the first update condition.

6. The method of claim 4, wherein the content acquisition device has a plurality of predetermined acquisition frame rates, the plurality of predetermined acquisition frame rates being respectively matched to a plurality of push frame rate ranges, and wherein controlling the frame rate of the content acquisition device to decrease from the first acquisition frame rate to the second acquisition frame rate comprises: In response to determining that the push parameters satisfy the first update condition, it is determined whether the push frame rate falls within the push frame rate range of the plurality of push frame rate ranges; as well as Based on the determination that the push frame rate falls within the second push frame rate range of the plurality of push frame rate ranges, the frame rate of the content acquisition device is controlled to decrease from the first acquisition frame rate to the second acquisition frame rate among the plurality of predetermined acquisition frame rates that matches the second push frame rate range.

7. The method of claim 1, wherein the push parameters include encoding parameters of an encoder, the encoder being used to encode the first content to obtain the first live content, and wherein controlling the frame rate of the content acquisition device to update from the first acquisition frame rate to the second acquisition frame rate based on the push parameters of the first live content includes: Based on the encoding parameters, it is determined whether the encoder has redundant encoding capability for the first content acquired at the first acquisition frame rate; Based on the determination that the encoder has redundant encoding capability for the first content, it is determined that the push parameter satisfies the second update condition for the frame rate of the content acquisition device. as well as In response to the push parameters satisfying the second update condition, the frame rate of the content acquisition device is controlled to increase from the first acquisition frame rate to the second acquisition frame rate.

8. The method according to claim 1, wherein controlling the content acquisition device to acquire the first content at a first acquisition frame rate comprises: Based on the configuration information of the content acquisition device, the frame rate of the content acquisition device is set to the first acquisition frame rate; as well as The content acquisition device is controlled to acquire the first content at the first acquisition frame rate.

9. A device for live streaming control, comprising: The first acquisition module is configured to control the content acquisition device to acquire the first content at a first acquisition frame rate during the live broadcast. The generation module is configured to generate first live content based on the collected first content for push to the client associated with the live stream; The update module is configured to control the frame rate of the content acquisition device to be updated from the first acquisition frame rate to the second acquisition frame rate based on the push parameters of the first live content. as well as The second acquisition module is configured to control the content acquisition device to acquire second content at the second acquisition frame rate for generating second live content.

10. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8 when executed by the at least one processor.

11. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

Images