Live broadcast method, device, electronic device and medium

Abstract

The present disclosure provides a live broadcast method, device, electronic device and medium, which relate to the field of computer technology, and in particular to the field of cloud phones, cloud computing and cloud service technology. The specific implementation scheme is: obtaining the host's audio and video data and target control instructions from the target client, and controlling the target cloud device according to the target control instructions to generate target video stream data; sending the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other clients display according to the target video stream data and the audio and video data. The present disclosure achieves the effect of reducing the requirements of live broadcast for client hardware, so that low-performance clients can still broadcast normally, thereby increasing the popularity of live broadcast.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of cloud mobile phones, cloud computing and cloud service technologies, and more particularly to a live broadcast method, device, electronic device and medium. Background Art

[0002] With the improvement of living standards, the penetration rate of smart phones is getting higher and higher, and the types of applications in smart phones are becoming more diverse, which has brought great convenience to people's lives. Live broadcast applications are a popular type of application at present. The anchor uses smart phones to show the audience a variety of live broadcast content.

[0003] Existing live broadcast technology requires the host's smartphone to render and encode the display screen, and then the server pushes the rendered and encoded video stream to the audience. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and medium for reducing the hardware requirements of live broadcast on the client.

[0005] According to one aspect of the present disclosure, a live broadcast method is provided, comprising:

[0006] Acquire the host's audio and video data and target control instructions from the target client, and control the target cloud device according to the target control instructions to generate target video stream data;

[0007] The target video stream data and the audio and video data are sent to the target client and at least one other client except the target client, so that the target client and the other clients are displayed according to the target video stream data and the audio and video data.

[0008] According to another aspect of the present disclosure, a live broadcast device is provided, comprising:

[0009] The video stream data generation module is used to obtain the host's audio and video data and target control instructions from the target client, and control the target cloud device according to the target control instructions to generate target video stream data;

[0010] The data sending module is used to send the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other clients display the target video stream data and the audio and video data.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method described in the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any one of the methods of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program executes any one of the methods of the present disclosure when a processor is used.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0019] Figure 1 is a flow chart of some live broadcast methods disclosed according to embodiments of the present disclosure;

[0020] Figure 2A is a flow chart of other live broadcast methods disclosed according to embodiments of the present disclosure;

[0021] Figure 2B It is a schematic diagram of the structure of some live broadcast systems disclosed according to the embodiments of the present disclosure;

[0022] Figure 3 It is a schematic diagram of the structure of some live broadcast devices disclosed according to the embodiments of the present disclosure;

[0023] Figure 4 It is a block diagram of an electronic device used to implement the live broadcast method disclosed in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] During the research and development process, the applicant found that the process of existing live broadcast technology can be summarized as follows: the anchor operates the live broadcast application running on his own smartphone, during which the smartphone renders and generates a screen display, the smartphone encodes the screen display to obtain video stream data, the smartphone sends the video stream data to the server, the server then sends the video stream data to other smartphones connected to the server, and other smartphones decode the video stream data. The audience can watch the screen display of the anchor's smartphone through other smartphones, thereby realizing the live broadcast of the live broadcast application.

[0026] However, existing live broadcasting technologies require the host's smartphone to render images to generate screen display images, and to perform video encoding and decoding on the screen display images to generate video stream data. This undoubtedly places high hardware requirements on the host's smartphone, resulting in low-performance smartphones experiencing freezes or image distortion during the live broadcast process, making it impossible to broadcast live normally.

[0027] Figure 1 This is a flowchart of some live broadcast methods disclosed in the embodiments of the present disclosure. This embodiment can be applied to the case where the anchor broadcasts live broadcasts through a cloud device. The method of this embodiment can be executed by the live broadcast device disclosed in the embodiments of the present disclosure. The device can be implemented by software and / or hardware and can be integrated on any electronic device with computing capabilities.

[0028] like Figure 1 As shown, the live broadcast method disclosed in this embodiment may include:

[0029] S101, obtaining the host's audio and video data and target control instructions from the target client, and controlling the target cloud device according to the target control instructions to generate target video stream data.

[0030] The target client represents an intelligent terminal device, such as a smart phone, a smart watch, a tablet computer, a laptop computer, or any electronic device equipped with an intelligent operating system. The host represents the holder of the target client, and the host broadcasts live through the target client. The audio and visual data include the host's voice data and appearance data, wherein the audio data can be the real voice of the host collected, or it can be a synthesized voice generated by TTS (Text To Speech, TTS) technology, etc.; the appearance data can be the real appearance of the host collected, or it can be a virtual host image generated by character model technology, etc. This embodiment does not limit the host's audio and visual data.

[0031] Cloud devices refer to devices that use cloud computing technology on the server side and implement cloud services through the server side. They can achieve many functions through the network by relying on their own systems and the servers set up by the manufacturer. The types of cloud devices include but are not limited to cloud phones, cloud computers, cloud tablets, and cloud TVs, etc. This embodiment does not limit the specific types of cloud devices. In this embodiment, the server side is equipped with at least one cloud device, and each client corresponds to a cloud device with control permissions, and the target cloud device refers to the cloud device that the target client has control permissions.

[0032] In one embodiment, the anchor clicks on the cloud device application icon pre-installed in the target client, and the target client generates a connection request and sends it to the server. The server authenticates the target client and establishes a communication connection with the target client after the authentication is passed. Furthermore, the server controls the target cloud device to render the cloud device interface, encodes the rendered interface, generates video stream data, and sends it to the target client. After receiving the video stream data, the target client decodes the video to display the cloud device interface on the screen. Among them, when the target client logs in to the cloud device for the first time, the cloud device interface can be the desktop interface of the cloud device.

[0033] The anchor can apply control operations to the cloud device interface through the screen, and the target client determines the screen coordinates involved in the control operation according to the user's control operation, and generates a target control instruction containing the screen coordinates. This embodiment does not limit the specific form of the control operation, and any way of controlling the cloud device interface can be considered as a control operation on the cloud device interface.

[0034] For example, when the cloud device interface is a desktop interface, the anchor can click on any application icon in the desktop interface through the screen, and the target client generates a target control instruction containing the screen coordinates of the application to start the application corresponding to the application icon installed in the cloud device.

[0035] For another example, when the cloud device interface is a game interface of a game application, the host can perform pressing and dragging operations through the screen. The target client generates target control instructions containing screen coordinates based on the screen coordinates involved in the pressing and dragging operations to control the movement or attack of the game character.

[0036] For another example, when the cloud device interface is a teaching interface of an educational application, the host can swipe left or right on the screen to control the turning of pages of a textbook or presentation, or press and drag on the screen to write text or formulas, etc. The target client generates a target control instruction including screen coordinates based on the screen coordinates involved in the swipe left or right operation and the screen coordinates involved in the press and drag operation.

[0037] The target client collects the host's voice and shape data through the collection equipment installed by itself. The voice and shape data can be collected in real time or when the host applies control operations to the cloud device interface. Optionally, when the host applies control operations to the cloud device interface, the target client collects the user's real voice data through the installed sound pickup equipment, such as a microphone, and also collects the user's real appearance data through the installed image collection equipment, such as a video camera or a camera. The target client associates and packages the real voice data and the real appearance data to generate voice and shape data.

[0038] The target client sends the target control instruction and the audio and video data to the server. The server receives the audio and video data and the target control instruction, and controls the cloud device to perform corresponding control operations in the cloud device interface according to the screen coordinates carried in the target control instruction. The cloud device uses the graphics processor carried by it to render the cloud device interface to obtain the interface display content, and uses the video encoder carried by it to perform video encoding on the interface display content, generating target video stream data containing the interface display content.

[0039] By obtaining the host's audio and video data and target control instructions from the target client, and controlling the target cloud device according to the target control instructions, the target video stream data is generated. This achieves the effect of generating target video stream data based on the host's control operations on the target cloud device on the target client, laying the foundation for subsequent streaming.

[0040] S102: Send the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other clients display the target video stream data and the audio and video data.

[0041] In addition to the communication connection with the target client, the server also communicates with at least one other client. The other client also represents a smart terminal device, such as a smart phone, a smart watch, a tablet computer, a laptop computer, or any electronic device equipped with a smart operating system.

[0042] In one embodiment, the server sends the target video stream data to the target client, and the target client decodes the target video stream data to display the interface display content of the cloud device on the screen of the target client. In addition, the server sends the target video stream data and the audio and video data of the host to other clients, and the other clients decode the target video stream data and parse the audio and video data to display the interface display content of the cloud device and the appearance of the host on the screen of the other clients at the same time, and play the host's voice through the speakers of the other clients.

[0043] The present invention obtains the host's audio and video data and target control instructions from the target client, controls the target cloud device according to the target control instructions, generates target video stream data, and then sends the target video stream data and audio and video data to the target client and at least one other client except the target client, so that the target client and other clients display according to the target video stream data and audio and video data. Since the rendering and video encoding processes required for generating the video stream are all executed in the cloud device, the target client only needs to send the target control instructions to the server, and the data processing pressure is relatively small, thereby achieving the effect of reducing the requirements of the client hardware for live broadcast, so that low-performance clients can still broadcast live normally, thereby increasing the popularity of live broadcast; and, according to the target video stream data and the host's audio and video data, they are displayed simultaneously, thereby improving the vividness of the live broadcast and optimizing the live broadcast effect.

[0044] Based on the above embodiment, after "generating target video stream data" in S101, the following steps are further included:

[0045] The target video stream data and audio and video data are stored in association to generate live playback data.

[0046] In one embodiment, after the server generates target video stream data each time, it will associate and store the target video stream data and audio and video data in the server to generate live playback data. When the target client and / or other clients initiate a live playback request to the server, the server will send all the associated stored target video stream data and audio and video data to the target client and / or other clients, so that the target client and / or other clients can watch the playback of the live broadcast.

[0047] By associating and storing the target video stream data and the audio and video data, live playback data is generated, providing users with a live playback function, meeting the user's needs for watching live playback, and improving the user experience.

[0048] Figure 2A It is a flowchart of other live broadcast methods disclosed in the embodiments of the present disclosure, which is further optimized and expanded based on the above technical solution and can be combined with the above optional implementation methods.

[0049] like Figure 2A As shown, the live broadcast method disclosed in this embodiment may include:

[0050] S201. Obtain a control request for a target cloud device from a target client, and authenticate the target client based on identity information included in the control request.

[0051] Among them, identity information refers to a kind of identification information, including but not limited to passwords, client device codes, account passwords, etc.

[0052] In one embodiment, the target client generates a control request carrying identity information and sends the control request to the server. The server obtains the control request for the target cloud device from the target client and parses the control request to obtain the identity information. The server compares the identity information of the target cloud device with the pre-stored standard identity information to authenticate the target client.

[0053] S202: When the identity authentication is passed, the control authority for the target cloud device is issued to the target client.

[0054] In one embodiment, if it is determined that the identity information of the target cloud device is the same as the standard identity information, the target client is issued the control permission for the target cloud device. If it is determined that the identity information of the target cloud device is different from the standard identity information, the control permission is denied to the target client, and the target client is prompted to re-authenticate.

[0055] S203: Acquire the host's audio and video data and target control instructions from the target client, and acquire the first resolution and first coordinate information of the target client from the target control instructions.

[0056] The first coordinate information indicates the screen coordinates involved in the host's control operation on the cloud device interface. For example, if the host clicks on a point A in the cloud device interface, the screen coordinates of point A are used as the first coordinate information. The first resolution indicates the display resolution of the target client screen.

[0057] In one implementation, the target client generates a target manipulation instruction carrying the first resolution and the first coordinate information, and sends the target manipulation instruction to the server. The server parses the target manipulation instruction to obtain the first resolution and the first coordinate information.

[0058] S204. Determine the second coordinate information corresponding to the first coordinate information in the target cloud device according to the first resolution of the target client and the second resolution of the target cloud device, and control the target cloud device according to the second coordinate information to generate target video stream data.

[0059] The second resolution of the target cloud device represents the display resolution of the cloud device. Since the first coordinate information represents absolute coordinates, and usually the first resolution of the target client and the second resolution of the target cloud device are not the same, it is necessary to determine the second coordinate information corresponding to the first coordinate information in the target cloud device.

[0060] In one embodiment, a first ratio between the number of pixels of the first resolution in the X-axis direction and the X coordinate in the first coordinate information, and a second ratio between the number of pixels of the first resolution in the Y-axis direction and the Y coordinate in the first coordinate information are determined to determine the relative coordinates of the first coordinate information at the first resolution. Then, the absolute coordinate information is determined according to the relative coordinates and the second resolution of the target cloud device as the second coordinate information corresponding to the first coordinate information in the target cloud device. The server controls the cloud device to perform corresponding control operations in the cloud device interface according to the determined second coordinate information, and generates target video stream data.

[0061] S205: Send the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other clients display the target video stream data and the audio and video data.

[0062] The present disclosure achieves the effect of authenticating the target client by obtaining a control request for the target cloud device from the target client, and authenticating the target client according to the identity information contained in the control request, and issuing the control authority of the target cloud device to the target client if the identity authentication is passed, thereby ensuring that the control authority of the target client can be obtained only if the identity authentication is passed; by obtaining the first resolution and first coordinate information of the target client from the target control instruction, and determining the second coordinate information corresponding to the first coordinate information in the target cloud device according to the first resolution of the target client and the second resolution of the target cloud device, and then controlling the target cloud device according to the second coordinate information to generate target video stream data, thereby achieving the effect of controlling the target cloud device according to the second coordinate information after the relative coordinate conversion, ensuring the accuracy of controlling the target cloud device, and avoiding misoperation of the target cloud device.

[0063] Based on the above embodiment, the method further includes:

[0064] A permission request for the target cloud device is obtained from the target client, and at least one auxiliary client is determined from other clients according to the client identifier included in the permission request, and the control permission for the target cloud device is issued to the auxiliary client.

[0065] The client identifier includes but is not limited to unique identification information such as the client device number or name.

[0066] In one embodiment, the target client generates a permission issuance request carrying a client identifier, and sends the permission issuance request to the server. The server parses the permission issuance request to obtain at least one client identifier, and uses at least one other client corresponding to the at least one client identifier as an auxiliary client, and then issues the control permission of the target cloud device to the auxiliary client, so that the user of the auxiliary client can operate and control the target cloud device.

[0067] By issuing control rights to the target cloud device to the auxiliary client, the user of the auxiliary client can also control the target cloud device, which improves the interactivity of the live broadcast. For example, in a teaching scenario, students can use the auxiliary client to control the target cloud device to interact with the teacher, such as asking or answering questions; in another example, in a game scenario, the audience can use the auxiliary client to control the target cloud device and play games with the game host on the same stage.

[0068] On the basis of the above embodiment, after issuing the control authority of the target cloud device to the auxiliary client, the method further includes:

[0069] The auxiliary control instruction is obtained from the auxiliary client, and the target cloud device is controlled according to the auxiliary control instruction to generate auxiliary video stream data, and then the auxiliary video stream data is sent to the auxiliary client, the target client and other clients except the auxiliary client.

[0070] In one embodiment, the user of the auxiliary client can apply control operations to the target cloud device interface through the screen. The auxiliary client determines the screen coordinates involved in the control operation based on the user's control operation, and generates auxiliary control instructions containing the screen coordinates. The server obtains the auxiliary control instructions from the auxiliary client, and controls the cloud device to perform corresponding control operations in the cloud device interface based on the parsed screen coordinates to generate auxiliary video stream data. The server sends the auxiliary video stream data to the auxiliary client, the target client, and other clients except the auxiliary client, for video decoding of the auxiliary video stream data to display the target cloud device interface.

[0071] By obtaining auxiliary control instructions from the auxiliary client and controlling the target cloud device according to the auxiliary control instructions, auxiliary video stream data is generated, and then the auxiliary video stream data is sent to the auxiliary client, the target client and other clients except the auxiliary client, so that the user of the auxiliary client can also control the target cloud device, thereby improving the interactivity of the live broadcast.

[0072] Based on the above embodiment, the method further includes:

[0073] Obtain other control instructions from at least one other client, control at least one other cloud device according to the other control instructions, generate at least one other video stream data, generate video stream summary data according to at least one other video stream data, and then send at least one other video stream data to at least one other client, and send the video stream summary data to the target client.

[0074] Among them, other cloud devices refer to cloud devices that other clients have control permissions for. One other cloud device corresponds to one other client.

[0075] In one implementation, users of other clients can exert control operations on other cloud device interfaces through the screen, and other clients determine the screen coordinates involved in the control operations based on the user's control operations, and generate other control instructions containing the screen coordinates. The server obtains other control instructions from other clients, and controls other cloud devices to perform corresponding control operations in the cloud device interface based on the parsed screen coordinates, generating other video stream data.

[0076] The server aggregates at least one other video stream data into one video stream aggregate data. The server sends at least one other video stream data to corresponding other clients respectively, and sends the video stream aggregate data to the target client. The other clients are displayed according to the corresponding other video stream data respectively, and the target client is displayed according to the video stream aggregate data.

[0077] For example, in a teaching scenario, assuming that students use other clients to control other corresponding cloud devices to answer questions, other clients will send other video stream data generated based on the answering interface of other cloud devices to other clients corresponding to other cloud devices, so that students can watch the answering interface of other cloud devices through the screens of other clients. For example, if student A controls other cloud device A through other client A to answer questions, then other cloud device A generates other video stream data and sends it to other client A, and student A watches the answering interface of other cloud device A through the screen of other client A.

[0078] Furthermore, the teacher is a user of the target client. The server aggregates at least one other video stream data generated by at least one other cloud device to generate video stream aggregate data, and sends the video stream aggregate data to the target client, so that the teacher can watch the answering interface of each student corresponding to the other cloud device through the screen of the target client.

[0079] By obtaining other control instructions from at least one other client and controlling at least one other cloud device according to the other control instructions, at least one other video stream data is generated, video stream summary data is generated according to at least one other video stream data, and then at least one other video stream data is sent to at least one other client, and the video stream summary data is sent to the target client, the effect of synchronously displaying the interfaces of other cloud devices on the target client is achieved, which is suitable for scenarios such as invigilation or monitoring.

[0080] Figure 2B is a schematic diagram of the structure of some live broadcast systems disclosed in the embodiments of the present disclosure, such as Figure 2B As shown, the live broadcast system includes a server 200, in which a target cloud device 201 and at least one other cloud device 202 are built. The live broadcast system also includes a target client 203 and at least one other client 204, and the target client 203 and at least one other client 204 are communicatively connected to the server 200.

[0081] In this embodiment, the type of the server 200 is an ARM structure server. In order to meet the needs of multiple users running multiple services at the same time, the platform layer of the traditional X86 structure server needs to perform virtualized container isolation design on the instance. However, since the server 200 adopts the ARM structure, each ARM motherboard can only be connected to one user at a time, and there is no need to perform security isolation of a single ARM motherboard, which simplifies the structural design scheme.

[0082] The server 200 supports multiple operating system platform ecosystems, including but not limited to Android, Ubuntu and other Linux kernel-based systems, and also supports Windows 11 systems.

[0083] The server 200 also has rich software resource package adaptation support, including but not limited to json, ext3, vfat, ntfs, iso9660, jffs and c and other file types.

[0084] Figure 3 This is a schematic diagram of the structure of some live broadcast devices disclosed in the embodiments of the present disclosure, which can be applicable to the situation where the anchor broadcasts live broadcasts through cloud devices. The device of this embodiment can be implemented by software and / or hardware, and can be integrated on any electronic device with computing capabilities.

[0085] like Figure 3 As shown, the live broadcast device 30 disclosed in this embodiment may include a video stream data generating module 31 and a data sending module 32, wherein:

[0086] The video stream data generating module 31 is used to obtain the host's audio and video data and target control instructions from the target client, and control the target cloud device according to the target control instructions to generate target video stream data;

[0087] The data sending module 32 is used to send the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other clients display the target video stream data and the audio and video data.

[0088] Optionally, the video stream data generating module 31 is specifically used for:

[0089] Acquire the first resolution and first coordinate information of the target client from the target manipulation instruction;

[0090] Determine, according to the first resolution of the target client and the second resolution of the target cloud device, second coordinate information corresponding to the first coordinate information in the target cloud device;

[0091] The target cloud device is controlled according to the second coordinate information to generate target video stream data.

[0092] Optionally, a video stream distribution module is also included, which is specifically used for:

[0093] Obtaining other control instructions from the at least one other client, and controlling at least one other cloud device according to the other control instructions to generate at least one other video stream data;

[0094] Generate video stream summary data according to the at least one other video stream data;

[0095] The at least one other video stream data is sent to the at least one other client, and the video stream summary data is sent to the target client.

[0096] Optionally, a data storage module is also included, which is specifically used for:

[0097] The target video stream data and the audio and video data are associated and stored to generate live playback data.

[0098] Optionally, an authentication module is also included, specifically for:

[0099] Obtaining a control request for the target cloud device from a target client, and authenticating the target client according to identity information included in the control request;

[0100] When the identity authentication is passed, the control authority for the target cloud device is issued to the target client.

[0101] Optionally, it also includes a permission issuance module, which is specifically used to:

[0102] Obtaining a permission issuance request for the target cloud device from the target client, and determining at least one auxiliary client from the other clients according to a client identifier included in the permission issuance request;

[0103] The control authority for the target cloud device is issued to the auxiliary client.

[0104] Optionally, an auxiliary control module is also included, which is specifically used for:

[0105] Acquire auxiliary control instructions from the auxiliary client, and control the target cloud device according to the auxiliary control instructions to generate auxiliary video stream data;

[0106] The auxiliary video stream data is sent to the auxiliary client, the target client and other clients except the auxiliary client.

[0107] The live broadcast device 30 disclosed in the embodiment of the present disclosure can execute the live broadcast method disclosed in the embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method. The contents not described in detail in this embodiment can refer to the description in the embodiment of the method of the present disclosure.

[0108] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0109] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0110] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0111] like Figure 4As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0112] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0113] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as the live broadcast method. For example, in some embodiments, the live broadcast method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the live broadcast method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the live broadcast method in any other appropriate manner (e.g., by means of firmware).

[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0116] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0119] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0121] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A live broadcast method, comprising: Authentication of the target client is performed, and a communication connection with the target client is established after the authentication is passed; Control the target cloud device to render the cloud device interface, perform video encoding on the rendered interface, generate video stream data and send it to the target client, so that the target client performs video decoding after receiving the video stream data to display the cloud device interface on the screen; the target cloud device represents the cloud device that the target client has control authority, and the cloud device is a cloud computer, a cloud tablet and a cloud TV; Acquire the host's audio and video data and target control instructions from the target client, and acquire the first resolution and first coordinate information of the target client from the target control instructions; the first coordinate information indicates the screen coordinates involved in the host's control operation on the cloud device interface; Determine, according to the first resolution of the target client and the second resolution of the target cloud device, the second coordinate information corresponding to the first coordinate information in the target cloud device, and control the target cloud device according to the second coordinate information to generate target video stream data; Sending the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other client display the target video stream data and the audio and video data; Among them, it also includes: Sending other video stream data generated based on the answering interface of other cloud devices to other clients corresponding to other cloud devices, so that students can watch the answering interface of other cloud devices through the screens of other clients; At least one other video stream data generated by at least one other cloud device is aggregated to generate video stream aggregate data, and the video stream aggregate data is sent to the target client, so that the teacher can watch the answering interface of each student corresponding to other cloud devices through the screen of the target client.

2. The method according to claim 1, further comprising: Obtaining other control instructions from the at least one other client, and controlling at least one other cloud device according to the other control instructions to generate at least one other video stream data; Generate video stream summary data according to the at least one other video stream data; The at least one other video stream data is sent to the at least one other client, and the video stream summary data is sent to the target client.

3. The method according to claim 1, after generating the target video stream data, further comprising: The target video stream data and the audio and video data are associated and stored to generate live playback data.

4. The method according to claim 1, before obtaining the host's audio and video data and target control instructions from the target client, further comprises: Obtaining a control request for the target cloud device from a target client, and authenticating the target client according to identity information included in the control request; When the identity authentication is passed, the control authority for the target cloud device is issued to the target client.

5. The method according to claim 4, further comprising: Obtaining a permission issuance request for the target cloud device from the target client, and determining at least one auxiliary client from the other clients according to a client identifier included in the permission issuance request; The control authority for the target cloud device is issued to the auxiliary client.

6. The method according to claim 5, after issuing the control permission of the target cloud device to the auxiliary client, further comprises: Acquire auxiliary control instructions from the auxiliary client, and control the target cloud device according to the auxiliary control instructions to generate auxiliary video stream data; The auxiliary video stream data is sent to the auxiliary client, the target client and other clients except the auxiliary client.

7. A live broadcast device, comprising: The video stream data generation module is used to authenticate the target client and establish a communication connection with the target client after the identity authentication is passed; Control the target cloud device to render the cloud device interface, perform video encoding on the rendered interface, generate video stream data and send it to the target client, so that the target client performs video decoding after receiving the video stream data to display the cloud device interface on the screen; the target cloud device represents the cloud device that the target client has control authority, and the cloud device is a cloud computer, a cloud tablet and a cloud TV; Acquire the host's audio and video data and target control instructions from the target client, and acquire the first resolution and first coordinate information of the target client from the target control instructions; the first coordinate information indicates the screen coordinates involved in the host's control operation on the cloud device interface; Determine, according to the first resolution of the target client and the second resolution of the target cloud device, the second coordinate information corresponding to the first coordinate information in the target cloud device, and control the target cloud device according to the second coordinate information to generate target video stream data; A data sending module, used for sending the target video stream data and the audio and video data to the target client and at least one other client except the target client, so that the target client and the other client display the target video stream data and the audio and video data; Among them, it also includes a video stream distribution module, which is specifically used for: Sending other video stream data generated based on the answering interface of other cloud devices to other clients corresponding to other cloud devices, so that students can watch the answering interface of other cloud devices through the screens of other clients; At least one other video stream data generated by at least one other cloud device is aggregated to generate video stream aggregate data, and the video stream aggregate data is sent to the target client, so that the teacher can watch the answering interface of each student corresponding to other cloud devices through the screen of the target client.

8. The device according to claim 7, wherein the video stream distribution module is further configured to: Obtaining other control instructions from the at least one other client, and controlling at least one other cloud device according to the other control instructions to generate at least one other video stream data; Generate video stream summary data according to the at least one other video stream data; The at least one other video stream data is sent to the at least one other client, and the video stream summary data is sent to the target client.

9. The device according to claim 7, further comprising a data storage module, specifically used for: The target video stream data and the audio and video data are associated and stored to generate live playback data.

10. The device according to claim 7, further comprising an identity authentication module, specifically configured to: Obtaining a control request for the target cloud device from a target client, and authenticating the target client according to identity information included in the control request; When the identity authentication is passed, the control authority for the target cloud device is issued to the target client.

11. The device according to claim 10, further comprising a permission issuing module, specifically configured to: Obtaining a permission issuance request for the target cloud device from the target client, and determining at least one auxiliary client from the other clients according to a client identifier included in the permission issuance request; The control authority for the target cloud device is issued to the auxiliary client.

12. The device according to claim 11, further comprising an auxiliary control module, specifically used for: Acquire auxiliary control instructions from the auxiliary client, and control the target cloud device according to the auxiliary control instructions to generate auxiliary video stream data; The auxiliary video stream data is sent to the auxiliary client, the target client and other clients except the auxiliary client.

13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

Images