A live broadcast control method, system and related device

Abstract

The application discloses a live broadcast control method and system and related equipment, and relates to the technical field of Internet live broadcast. The application responds to a live broadcast access request issued by a user, acquires a rendering state detection result of a first video frame of a client, and acquires a transmission state detection result of a corresponding live broadcast stream of a server; in the case that the rendering state detection result and the transmission state detection result both meet preset allowed access conditions, a transmission link of the corresponding live broadcast stream between the server is established. According to the application, double-end state consistency verification in a live broadcast starting stage can be realized, the phenomenon of invalid live broadcast starting is effectively avoided, and the success rate of live broadcast starting and user experience are improved.

Description

Technical Field

[0001] This application relates to the field of Internet live streaming technology, and more specifically, to a live streaming control method, system and related equipment. Background Technology

[0002] With the rapid development of internet technology, live streaming has become an important application form in fields such as information dissemination, online education, and entertainment interaction. The availability of audio and video during the live streaming launch phase is the core foundation for ensuring the quality of live streaming services, and directly determines the core user experience and the commercial value of the live streaming business.

[0003] To enable rapid startup of live streaming services, the decision-making logic during the live streaming startup phase is generally based on lightweight decision-making mechanisms at the network and protocol layers. Specifically, for the push streaming end, a connection using transport protocols such as RTMP or RTSP is typically established between the push streaming client and the live streaming server. After receiving a successful handshake response from the server, the push streaming is deemed ready and the live streaming begins. For the pull streaming end, the pull streaming is deemed ready and the live streaming playback interface is directly loaded after the pull streaming client sends a pull request to the server, obtains the live stream address, and establishes a TCP transport connection.

[0004] However, the aforementioned lightweight judgment mechanism only relies on superficial states such as network layer connection establishment or protocol layer handshake success, and cannot deeply verify the actual availability of the live audio and video streams. This can easily lead to situations where the live interface is loaded but the audio and video cannot be output normally, resulting in the live stream starting up being invalid. Summary of the Invention

[0005] In view of the above problems, this application is made to provide a live streaming control method, system, and related equipment to determine the validity of live streaming initiation. The specific solution is as follows: Firstly, this application provides a live streaming control method, applied to a client, including: In response to the live streaming access request sent by the client, the server obtains the rendering status detection result of the client on the initial video frame to be rendered, and the server obtains the transmission status detection result of the corresponding live streaming stream. If both the rendering status detection result and the transmission status detection result meet the preset allowed access conditions, a transmission link for the corresponding live stream is established between the server and the server.

[0006] In one possible design, in another implementation of the first aspect of this application, when the live stream interaction mode requested by the live stream access request is a push stream mode, the process of obtaining the rendering status detection result of the client for the initial video frame to be rendered includes: Create a local preview player and render the first video frame captured from the client's camera using the local preview player; If the local preview player does not output a rendered image within a preset time, or if the output rendered image does not meet the preset rendering conditions, then the rendering status detection result corresponding to the client is determined to be rendering failure. If the rendered image output by the local preview player meets the preset rendering conditions within a preset time, then the rendering status detection result corresponding to the client is determined to be successful rendering.

[0007] In one possible design, in another implementation of the first aspect of this application, when the live stream interaction mode requested by the live stream access request is a pull stream mode, the process of obtaining the rendering status detection result of the client for the initial video frame to be rendered includes: Create a temporary test player and establish a temporary streaming connection based on the streaming address of the desired live stream room to retrieve the current live stream data of the desired live stream room; The temporary test player decodes and renders the live stream data. If the temporary test player does not output a rendered image within a preset time, or if the output rendered image does not meet the preset rendering conditions, then the rendering status detection result corresponding to the client is determined to be rendering failure. If the rendered image output by the temporary test player meets the preset rendering conditions within a preset time, the rendering status detection result corresponding to the client is determined to be successful.

[0008] In one possible design, in another implementation of the first aspect of this application, when the live stream interaction mode requested by the live stream access request is a push stream mode, the process of obtaining the transmission status detection result of the corresponding live stream on the server includes: Send a live stream registration request to the server and obtain the live stream identifier returned by the server; Send the live stream attribute information to the server to query the live stream corresponding to the live stream identifier, and obtain the live stream attribute information of the live stream returned by the server; Based on the live stream attribute information, the transmission status detection result of the corresponding live stream on the server is determined.

[0009] In one possible design, in another implementation of the first aspect of this application, when the live stream interaction mode requested by the live stream access request is a pull stream mode, the process of obtaining the transmission status detection result of the corresponding live stream on the server includes: Send the live stream attribute information of the live stream to the server for querying the live stream of the desired live stream, and obtain the live stream attribute information of the live stream returned by the server; Based on the live stream attribute information, the transmission status detection result of the corresponding live stream on the server is determined.

[0010] In one possible design, in another implementation of the first aspect of the embodiments of this application, the live stream attribute information includes metadata information and live stream readiness status, wherein the live stream readiness status is ready and not ready, and the metadata information includes live stream identifier validity, current bitrate and number of connections; The process of determining the transmission status detection result of the corresponding live stream on the server based on the live stream attribute information includes: If the metadata information is complete and the live stream is ready, then the transmission status detection result of the corresponding live stream on the server is determined to be that the live stream was successfully established; otherwise, the transmission status detection result is that the live stream was not established.

[0011] In one possible design, in another implementation of the first aspect of the embodiments of this application, when the live stream interaction mode requested by the live stream access request is a push stream mode, before responding to the live stream access request issued by the client, the following is also included: Obtain the input live streaming configuration parameters and perform integrity verification on the live streaming configuration parameters, which include server address, streaming key, live streaming title, cover information and viewing permission settings; If the verification passes, a live streaming access request corresponding to the live streaming configuration parameters is generated and sent to the server. If the verification fails, output information to prompt you to correct the configuration parameters.

[0012] In one possible design, in another implementation of the first aspect of the embodiments of this application, the method further includes: If either the rendering status detection result or the transmission status detection result fails to meet the allowed access conditions, the rendering status detection result and / or the transmission status detection result that fails to meet the allowed access conditions will be re-acquired until the number of retries reaches a preset threshold. Then, the process will revert to the state where the user issued the live streaming access request and output a prompt to detect the live streaming device.

[0013] Secondly, this application provides a live streaming control system, including: a client and a server, wherein the client is used to implement the live streaming control method described in any of the first aspects of this application above, so as to establish a transmission link between the client and the server for a corresponding live stream.

[0014] Thirdly, this application provides an electronic device, including: a memory and a processor; The memory is used to store programs; The processor is configured to execute the program to implement the live streaming control method described in any of the first aspects of this application.

[0015] By employing the aforementioned technical solution, the live streaming control method provided in this application, upon detecting a user's live streaming access request, utilizes a joint determination mechanism of client rendering status and server transmission status to perform parallel dual-end detection during the live streaming startup phase. It can be understood that the client-side first-frame rendering detection ensures the smooth operation of the local acquisition, decoding, and display links, while the server-side transmission status detection ensures the availability of the network transmission and stream distribution links. Only when both ends meet the conditions for allowing live streaming access is the corresponding live stream transmission link formally established between the client and the server. This dual-end detection mechanism fundamentally solves the problem of invalid live streaming startup that may result from single-end detection, effectively preventing black screens, no stream, or stuttering after users enter the live streaming room or start the live stream, thereby significantly improving the success rate of live streaming startup and user experience. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic diagram of the architecture of the live streaming control system provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a live streaming control method provided in an embodiment of this application; Figure 3 A flowchart illustrating another live streaming control method provided in an embodiment of this application; Figure 4 An example diagram of a live streaming control interface provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] This application can be widely applied to various live streaming scenarios involving real-time audio and video transmission. For example, in online education live streaming scenarios, the stability of the teacher's push stream directly affects whether the course can proceed smoothly, while the smoothness of the student's pull stream determines the quality of the learning experience; in e-commerce live streaming scenarios, a black screen on the host's end or the inability of viewers to watch normally will directly lead to a decrease in sales conversion rate; in corporate internal training or conference live streaming scenarios, the reliability of live streaming access is related to the timeliness and accuracy of information transmission; in entertainment and gaming live streaming scenarios, black screens or buffering after viewers enter the live stream will significantly reduce user retention rates. In these scenarios, the stability of the live streaming access phase is a core factor affecting user experience and business performance.

[0019] However, a common problem exists in the above application scenarios: the "false readiness" phenomenon easily occurs during live streaming access. This means the system determines the live stream is ready and the interface has loaded normally, but the actual audio and video streams are not truly usable, resulting in a black screen, no sound, or choppy playback for the user. For example, the streaming client may successfully render the first frame locally, thus determining the live stream is ready and entering the broadcast state, but the server may not actually receive valid stream data. In this case, the viewer will face a situation where there is no stream to watch. Alternatively, the server's stream registry may contain a record of the live stream, leading the system to determine that the live stream is ready and allowing the viewer to access it. However, due to decoder failure or network reception anomalies, the viewer's end may be unable to render a valid image, resulting in a black screen or choppy playback for the viewer.

[0020] Based on the actual needs in the above application scenarios and the shortcomings of existing technologies, this application proposes a live streaming control method. The live streaming control method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0021] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a client 100 and a server 200, wherein the server 200 may include one or more server-side components. Figure 1 (This example uses a server as an illustration).

[0022] The live streaming control method provided in this application is executed by client 100, where client 100 refers to an application that implements the live streaming function. This application can be either a self-developed live streaming application or an application that integrates a third-party live streaming SDK (Software Development Kit) to implement the live streaming function.

[0023] The specific device form that carries the above-mentioned application can be any of the following: mobile phone, tablet computer, large teaching screen, wearable device, in-vehicle device, conference terminal, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-large mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. This application embodiment does not impose any limitations on this.

[0024] This application provides a live streaming control method, which is illustrated by applying the method to a computer device. Specifically, the computer device may be... Figure 1 Client 100 in the middle. (Refer to...) Figure 2 The present application provides a flowchart of a live streaming control method, which may include steps S110 to S130. These steps are described in detail below.

[0025] Step S110: Respond to the live streaming access request sent by the client, obtain the rendering status detection result of the client for the initial video frame to be rendered, and obtain the transmission status detection result of the corresponding live stream on the server.

[0026] Among them, the live streaming access request refers to the trigger signal sent by the user, intelligent assistant, backend scheduling system, etc. on the client to establish a live streaming transmission link. The following explanation will take the user sending a live streaming access request on the client as an example.

[0027] It is understandable that this live stream access request can be either a start-up request sent by the user after selecting the push streaming mode on the push end, used to push local audio and video data to the server; or a watch request sent by the user after selecting the pull streaming mode on the pull end, used to pull audio and video data from the server for playback. After the client sends the live stream access request, it does not immediately establish a connection with the server, but instead triggers two parallel detection processes: the client's detection of the rendering status of the initial video frame to be rendered and the server's detection of the corresponding live stream transmission status.

[0028] The process of detecting the rendering status of the initial video frame to be rendered by the client can verify whether the client's audio and video processing link is normal. The initial video frame to be rendered refers to the first video frame data or the first N video frames data that are captured or received in the live stream. It is the basic carrier for determining whether the client can display the live content normally. If the client can successfully decode and render the initial video frame to be rendered, it means that the client's camera acquisition and encoding module (pull streaming scenario) or network reception and decoding module (pull streaming scenario) is working normally. Conversely, if the rendering fails, it means that there are hidden dangers such as black screen or decoding errors on the client's local side. If the live stream is forcibly connected in this case, the user will only see a black screen or a stuttering screen.

[0029] Meanwhile, obtaining the transmission status detection result of the corresponding live stream on the server side is a process of verifying whether the stream data distribution link on the server side is ready. The transmission status of the corresponding live stream on the server side refers to whether the server has successfully registered and maintained the metadata of the live stream (such as stream name, push and pull stream addresses, bitrate information, etc.). If the server finds that the live stream has been established and is in a ready state, it means that the network transmission channel has been successfully established; conversely, if the server has no record of the stream or is in a not-ready state, it means that the push end has not successfully pushed the stream or the stream has timed out. In this case, if the pull end connects, it will not be able to obtain data.

[0030] The aforementioned dual-end detection effectively identifies "false ready" states caused by inaccurate unilateral detection. For example, in certain network jitter scenarios, the client may briefly render successfully, but the server-side stream registry may have timed out and deleted the live stream. In this case, client-side detection alone would misjudge it as ready. Alternatively, the server-side stream record may exist, but the client-side decoder may crash and fail to render; server-side detection alone would also misjudge. Therefore, joint dual-end detection is crucial to ensuring successful live stream startup.

[0031] Step S120: Determine whether the rendering status detection result and the transmission status detection result both meet the preset allowed access conditions. If yes, proceed to step S130 to establish a transmission link between the server and the corresponding live stream. If not, return to step S110 to trigger a retry mechanism or report an error, thereby preventing forced entry into the live stream room when the link is down.

[0032] The preset access conditions refer to pre-defined logical standards used to determine whether a live stream is truly usable. Specifically, these conditions may require that the rendering status detection result indicates "rendering successful," and the transmission status detection result indicates "live stream established successfully." Only when both conditions are met simultaneously will the client determine that the current live streaming environment is ready. At this point, the client will proactively establish a stable live stream transmission link with the server. For example, for a push stream, the client will begin pushing locally collected audio and video data to the server; for a pull stream, the client will establish a pull connection with the server and begin pulling the live stream data from the desired live stream room.

[0033] Conversely, if either the rendering status detection result or the transmission status detection result fails to meet the allowed access conditions, such as rendering failure or live stream establishment failure, the client will prohibit the establishment of the transmission link and may further trigger a retry mechanism or report an error, thereby avoiding forcibly entering the live room when the link is unavailable.

[0034] In summary, the live streaming control method provided in this application, upon detecting a user's live streaming access request, employs a joint determination mechanism of client rendering status and server transmission status to perform parallel dual-end detection during the live streaming startup phase. It can be understood that the first-frame rendering detection on the client side ensures the smooth operation of the local acquisition, decoding, and display links, while the transmission status detection on the server side ensures the availability of the network transmission and stream distribution links. Only when both ends meet the conditions for allowing live streaming access is the corresponding live stream transmission link between the client and the server formally established. This dual-end detection mechanism fundamentally solves the problem of invalid live streaming startup that may result from single-end detection, effectively avoiding phenomena such as black screens, no stream, or stuttering after users enter the live streaming room or start the live stream, thereby significantly improving the success rate of live streaming startup and user experience.

[0035] Next, other possible implementations of the live streaming control method provided in this application will be described in detail through the following embodiments.

[0036] Understandably, a client's live streaming access request typically includes the selected live streaming interaction mode. Live streaming interaction modes can be divided into push streaming mode and pull streaming mode. In push streaming mode, the client acts as the push end, encoding the collected video and audio data and sending it to the designated server. In pull streaming mode, the client acts as the pull end, obtaining and playing the real-time video stream from the server.

[0037] Reference Figure 3 This application provides a flowchart of another live streaming control method, using the streaming end as an example, to introduce the complete live streaming process. The complete live streaming process can include the following states: initialization configuration, verification and submission, parallel detection, dual readiness, formal streaming, live streaming in progress, live streaming interruption, and live streaming end.

[0038] Specifically, before a broadcaster formally initiates a live stream access request, the client will first execute the steps of "initialization configuration" and "verification and submission." This may include: obtaining the input live stream configuration parameters; verifying the integrity of the live stream configuration parameters, which include the server address, streaming key, live stream title, cover information, and viewing permission settings; if the verification passes, generating a live stream access request corresponding to the live stream configuration parameters and sending the request to the server; if the verification fails, outputting information to prompt for correction of the configuration parameters.

[0039] Understandably, when a broadcaster starts a live stream, they will first enter the live stream control interface and select the push streaming mode, while viewers will select the pull streaming mode. The client receives the user's selection, initializes the process state, and displays the configuration items corresponding to the push streaming mode to the user, allowing the user to fill in the corresponding live stream configuration parameters. These live stream configuration parameters are the basic data support for the push streaming operation, and their specific content may include the server address (such as the push streaming address RTMP (Real Time Messaging Protocol) and the pull streaming playback address template), streaming key, live stream title, cover information, and viewing permission settings, etc.

[0040] Reference Figure 4 When displaying the above configuration items to the user on the client side, a "Cell + Model" approach can be used for data-driven interface design. Specifically, each live streaming configuration parameter (such as the push address, streaming key, live stream title, etc.) is abstracted into an independent Model object, encapsulating attributes such as parameter name, current value, placeholder hints, and whether it is required. Each Model corresponds to a unified Cell view component, responsible for displaying and interacting with that parameter. The client only needs to maintain a Model array to dynamically generate the complete configuration list. When the user modifies the configuration in a Cell, it is directly written back to the corresponding Model. Finally, by traversing the Model array, all the live streaming parameters entered by the user can be obtained. This approach allows for flexible addition and deletion of configuration items, maintains a consistent interface style, and facilitates parameter validation and subsequent reuse.

[0041] After the user completes the form, they will perform actions such as clicking "Start Live Stream" on the interface to indicate that the form is complete and the live stream will begin using the configured parameters. Further, the client retrieves the input live stream configuration parameters and performs integrity checks on them. The client verifies the parameters from multiple dimensions for this integrity check. For example, regarding the server address, the client checks whether it conforms to the standard URL (Uniform Resource Locator) format and whether the address is reachable; regarding the streaming key, the client checks whether it is empty, whether its length is within the preset range, and whether it contains illegal characters; regarding the live stream title and cover information, the client checks whether the title character count exceeds the limit, whether the cover image format is supported (e.g., JPG, PNG), and whether the image size exceeds the threshold; regarding viewing permission settings, the client checks whether the permission logic conflicts.

[0042] If the verification passes, the client treats the live streaming configuration parameters as valid data, generates a live streaming access request corresponding to the configuration parameters, and sends the request to the server. This means that only requests with complete and correctly formatted parameters will be sent, ensuring that subsequent rendering and transmission status checks can run under correct parameters, avoiding invalid checks and wasted network resources due to incorrect parameters. If the verification fails, the client outputs information to prompt users to correct the configuration parameters, such as "Server address format is incorrect, please check the protocol header" or "Streaming key cannot be empty," guiding the user to quickly locate and correct the problem and resubmit.

[0043] This application's embodiments establish the first line of defense for live streaming access by introducing a parameter integrity verification mechanism before the core live streaming process begins. By intercepting failures caused by configuration errors at the very beginning of the process, not only are invalid server requests reduced and system load lowered, but more importantly, the success rate of subsequent dual-end detection is significantly improved. This avoids users blindly waiting for rendering or transmission detection results when configurations are incorrect, greatly optimizing the user experience.

[0044] After the configuration parameters on the streaming end pass verification, the client responds to the live streaming access request and enters parallel detection state. The process of obtaining the client's rendering state detection results for the initial video frame to be rendered includes: First, the client creates a local preview player. It's important to note that this local preview player differs from the official playback interface shown to viewers; it primarily serves as a testing tool. The client uses this local preview player to render the first video frame captured from the client's camera, enabling preview testing of the video frames before live streaming.

[0045] If the local preview player fails to output a rendered image within a preset time (e.g., 3 seconds), or if the output rendered image does not meet the preset rendering conditions, the client's rendering status detection result is determined to be rendering failure. The preset rendering conditions refer to visual characteristic standards that prove the video stream is working normally. For example, preset rendering conditions may include a non-black screen, a non-placeholder image, a non-distorted screen, and a non-static, stuttering image. If the rendered image output by the local preview player meets the preset rendering conditions within the preset time, the client's rendering status detection result is determined to be rendering success, i.e., the client is ready.

[0046] It is understood that the embodiments of this application use local loopback detection to verify whether the entire link of client acquisition, encoding and decoding is smooth, so as to detect local client problems such as camera being occupied, encoder abnormality or graphics card driver failure in time before the streaming starts, and avoid black screen live broadcast accidents caused by blind streaming.

[0047] Meanwhile, in push streaming mode, the process of obtaining the transmission status detection results of the corresponding live stream on the server focuses on the allocation of server resources and the confirmation of link status. The specific steps are as follows: The client sends a live stream registration request to the server and obtains the live stream identifier returned by the server. This live stream identifier (such as the Stream Name or Key in the RTMP URL) is a unique identity credential assigned by the server for this live stream session, indicating that the server has reserved a push stream access point for this broadcaster. Subsequently, the client sends a query to the server to retrieve the live stream attribute information corresponding to the live stream identifier, and obtains the live stream attribute information returned by the server. This further confirms whether the live stream status on the server side is truly available. Finally, based on the live stream attribute information, the client determines the transmission status detection result of the corresponding live stream on the server.

[0048] Specifically, the live stream attribute information includes metadata information and live stream readiness status. The live stream readiness status has two states: ready and not ready. The metadata information includes key parameters such as the validity of the live stream identifier, the current bitrate, and the number of connections. If the metadata information is complete and the live stream readiness status is ready, the server-side transmission status detection result for the corresponding live stream indicates that the live stream has been successfully established; otherwise, the transmission status detection result indicates that the live stream has failed to establish. For example, if the metadata information returned by the server shows a bitrate of 0 or a number of connections of 0, or an invalid live stream identifier, it indicates that the stream link on the server side has not been successfully established. In this case, if the stream is forcibly pushed, the data packets will not be correctly distributed.

[0049] Through the aforementioned dual detection mechanism, a complete detection system for live streaming scenarios is constructed. Only when the local preview player renders successfully and the server-side stream attributes show that it is ready will the client establish a transmission link with the server for the corresponding live stream and officially begin streaming. This effectively solves the "false readiness" problem caused by single points of failure in existing technologies, ensuring the stability of the streamer's streaming.

[0050] In another possible implementation, where the live stream interaction mode requested by the live stream access request is pull streaming mode, the client is typically the viewer, used to pull and play live stream data from the server. The corresponding process of obtaining the client's rendering status detection result for the initial video frame to be rendered can include: First, the client creates a temporary test player and establishes a temporary streaming connection based on the streaming address of the desired live stream room, retrieving the current live stream data. The temporary test player differs fundamentally from the official player used by the audience. It runs in the background, its main function being to pre-detect the streaming link; its lifespan only lasts until the detection is complete, without consuming excessive system resources or affecting the user interface layout. The official player, on the other hand, is used to display a longer live stream to the user after the detection passes.

[0051] By establishing a temporary pull connection, the client can pre-acquire a small segment of live stream data, providing a data foundation for subsequent decoding, rendering, and detection. This process does not require sending a live stream registration request to the server as in push streaming mode; instead, it only needs to initiate a pull connection request via the pull address. This is because, in pull streaming mode, the live stream already exists on the server, and the viewer only needs to confirm whether the stream can be successfully pulled and played.

[0052] Subsequently, the client decodes and renders the acquired live stream data using a temporary test player to verify the client's local decoding capabilities and the effectiveness of the network downlink. If the temporary test player fails to output a rendered image within a preset time, or if the output rendered image does not meet the preset rendering conditions, it indicates that the client has insufficient local decoding capabilities or abnormal acquisition of live stream data. In this case, the client's rendering status detection result can be determined as rendering failure.

[0053] The preset rendering conditions are the visual standards for judging whether a live stream is normal. These can include a non-black screen, a non-distorted screen, and a non-static, stuttering display. For example, if the temporary test player continuously outputs a black screen for 3 seconds, it indicates that the live stream data may have decoding errors or data packet loss, and in this case, rendering is considered a failure. Conversely, if the rendered image output by the temporary test player meets the preset rendering conditions within the preset time, the client's corresponding rendering status detection result is determined to be successful.

[0054] Based on this, this application, through the pre-rendering mechanism of the temporary test player, can detect playback failures caused by network fluctuations or decoder anomalies before the user officially enters the live broadcast room.

[0055] Meanwhile, in pull streaming mode, the process of obtaining the transmission status detection results of the corresponding live stream on the server can include: The client sends live stream attribute information to the server to query the live stream of the desired live room, and receives the live stream attribute information in return from the server. Understandably, since the live stream has already been pushed from the broadcaster's client to the server, the server maintains the real-time status information of the live stream. The client can quickly obtain this information through the query interface without going through the registration process of the push streaming mode. Next, based on the live stream attribute information, the client determines the server's transmission status detection result for the corresponding live stream.

[0056] As described above, the live stream attribute information includes metadata information and live stream readiness status. If the metadata information is complete and the live stream readiness status is "ready," then the server-side transmission status detection result for the corresponding live stream indicates that the live stream has been successfully established; otherwise, the transmission status detection result indicates that the live stream has failed to be established.

[0057] Through the aforementioned dual detection mechanism, a dual-end detection system is constructed for streaming scenarios. Only when the temporary test player renders successfully and the server-side live stream is successfully established will the client establish a transmission link with the corresponding live stream on the server and switch to the official player for playback. This effectively solves the problem of viewers facing a black screen or loading failure after entering the live stream, significantly improving the viewer's experience.

[0058] It is understandable that in practical applications, the live streaming environment is complex and ever-changing. Network jitter, momentary codec stuttering, or server load fluctuations can all lead to single-pass detection failures. If the process is directly terminated and a failure is determined in step S120, it will reduce the success rate of live streaming access and affect user experience. Therefore, this application embodiment introduces a retry and rollback mechanism.

[0059] Specifically, if either the rendering state detection result or the transmission state detection result fails to meet the allowed access conditions, the client will not immediately terminate the process. Instead, it will trigger retry logic to re-acquire the rendering state detection result and / or transmission state detection result that fail to meet the allowed access conditions, until the number of retries reaches a preset threshold. Then, it will fall back to the state where the user issued the live streaming access request and output a prompt to detect the live streaming device.

[0060] For example, if the rendering status detection result indicates rendering failure, the client can destroy the current preview player instance, recreate a new local preview player, and restart the rendering detection process to rule out detection failure caused by errors in the player instance's internal state. Similarly, if the transmission status detection result indicates live stream establishment failure, the client can resend the query request to the server or re-initiate the live stream registration request to obtain a new stream identifier, thereby addressing any momentary resource scheduling delays on the server side.

[0061] However, unlimited retries can lead to a system loop, consuming excessive CPU and network resources, and even causing applications to become unresponsive. To address this issue, this embodiment sets a preset threshold. This preset threshold can be configured according to actual business needs, for example, set to 3 or 5 times. The system records the number of retries. When the number of retries has not reached the preset threshold, the retry logic continues to execute; until the number of retries reaches the preset threshold, the system determines that the current fault cannot be recovered through automatic retries, at which point a rollback operation is performed.

[0062] Here, rollback operation refers to the internal state machine logic of the system being rolled back. (See reference...) Figure 3 Taking the streaming client as an example, the complete live streaming process includes the following states: initial configuration, verification and submission, parallel detection, dual readiness, formal streaming (live streaming in progress, live streaming interrupted), and live streaming end. When the number of retries is exhausted and the results of the dual-end detection still do not meet the allowed access conditions, the system state will be forcibly rolled back to the "initial configuration" or "verification and submission" state, that is, the state before the user clicks the "start live streaming" button. At the same time, the client will output information prompting the detection of the live streaming device, guiding the user to conduct targeted troubleshooting and improving the efficiency of problem solving.

[0063] Similarly, if an anomaly occurs at any stage of the live stream, the system can at least revert to the previous stage, providing an opportunity for automatic repair of momentary faults. For example, if a stream anomaly or permission verification failure is detected during the live stream, the system automatically rolls back to the verification and submission state to ensure process integrity. When the live stream ends, in response to the user's click to end the stream, the client calls the server-side interaction module to close the session, stop streaming, and release player module resources, completing the entire closed loop.

[0064] This application also provides an electronic device in its embodiments. (See reference...) Figure 5 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, tablets, large-screen teaching displays, wearable devices, etc. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0065] like Figure 5As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the live broadcast control method of the foregoing embodiments of this application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.

[0066] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0067] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the live broadcast control methods provided in this application.

[0068] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the live broadcast control methods provided in this application.

[0069] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0071] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0072] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0073] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A live streaming control method, characterized in that, Applied to the client side, including: In response to the live streaming access request sent by the client, the server obtains the rendering status detection result of the client on the initial video frame to be rendered, and the server obtains the transmission status detection result of the corresponding live streaming stream. If both the rendering status detection result and the transmission status detection result meet the preset allowed access conditions, a transmission link for the corresponding live stream is established between the server and the server.

2. The live streaming control method according to claim 1, characterized in that, When the live stream interaction mode requested in the live stream access request is push streaming mode, the process of obtaining the rendering status detection result of the client for the initial video frame to be rendered includes: Create a local preview player and render the first video frame captured from the client's camera using the local preview player; If the local preview player does not output a rendered image within a preset time, or if the output rendered image does not meet the preset rendering conditions, then the rendering status detection result corresponding to the client is determined to be rendering failure. If the rendered image output by the local preview player meets the preset rendering conditions within a preset time, then the rendering status detection result corresponding to the client is determined to be successful rendering.

3. The live streaming control method according to claim 1, characterized in that, When the live stream interaction mode requested in the live stream access request is pull stream mode, the process of obtaining the rendering status detection result of the client for the initial video frame to be rendered includes: Create a temporary test player and establish a temporary streaming connection based on the streaming address of the desired live stream room to retrieve the current live stream data of the desired live stream room; The temporary test player decodes and renders the live stream data. If the temporary test player does not output a rendered image within a preset time, or if the output rendered image does not meet the preset rendering conditions, then the rendering status detection result corresponding to the client is determined to be rendering failure. If the rendered image output by the temporary test player meets the preset rendering conditions within a preset time, the rendering status detection result corresponding to the client is determined to be successful.

4. The live streaming control method according to claim 1, characterized in that, When the live stream interaction mode requested in the live stream access request is push streaming mode, the process of obtaining the transmission status detection result of the corresponding live stream on the server includes: Send a live stream registration request to the server and obtain the live stream identifier returned by the server; Send the live stream attribute information to the server to query the live stream corresponding to the live stream identifier, and obtain the live stream attribute information of the live stream returned by the server; Based on the live stream attribute information, the transmission status detection result of the corresponding live stream on the server is determined.

5. The live streaming control method according to claim 1, characterized in that, When the live stream interaction mode requested in the live stream access request is pull stream mode, the process of obtaining the transmission status detection result of the corresponding live stream on the server includes: Send the live stream attribute information of the live stream to the server for querying the live stream of the desired live stream, and obtain the live stream attribute information of the live stream returned by the server; Based on the live stream attribute information, the transmission status detection result of the corresponding live stream on the server is determined.

6. The live streaming control method according to any one of claims 4-5, characterized in that, The live stream attribute information includes metadata information and live stream readiness status, which is either ready or not ready. The metadata information includes the validity of the live stream identifier, the current bitrate, and the number of connections. The process of determining the transmission status detection result of the corresponding live stream on the server based on the live stream attribute information includes: If the metadata information is complete and the live stream is ready, then the transmission status detection result of the corresponding live stream on the server is determined to be that the live stream has been successfully established. Otherwise, the transmission status detection result indicates that the live stream establishment failed.

7. The live streaming control method according to claim 1, characterized in that, If the live stream interaction mode requested in the live stream access request is push streaming mode, the method further includes the following before responding to the live stream access request sent by the client: Obtain the input live streaming configuration parameters and perform integrity verification on the live streaming configuration parameters, which include server address, streaming key, live streaming title, cover information and viewing permission settings; If the verification passes, a live streaming access request corresponding to the live streaming configuration parameters is generated and sent to the server. If the verification fails, output information to prompt you to correct the configuration parameters.

8. The live streaming control method according to any one of claims 1-5 and 7, characterized in that, The method further includes: If either the rendering status detection result or the transmission status detection result fails to meet the allowed access conditions, the rendering status detection result and / or the transmission status detection result that fails to meet the allowed access conditions will be re-acquired until the number of retries reaches a preset threshold. Then, the process will revert to the state where the user issued the live streaming access request and output a prompt to detect the live streaming device.

9. A live streaming control system, characterized in that, include: The client and server are configured to implement the live streaming control method according to any one of claims 1-8, so as to establish a transmission link between the client and the server for the corresponding live stream.

10. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the live broadcast control method as described in any one of claims 1 to 8.

Images