Video stream distribution system and method, cloud server and storage medium

Abstract

The embodiment of the invention discloses a video stream distribution system and method, a cloud server and a storage medium. The video stream distribution system comprises a plurality of user interaction ends, a plurality of edge computing ends and at least one cloud server, wherein the cloud server is used for updating original user structure information corresponding to a user identifier in a user structure data queue into target user structure information based on user interaction information when determining that a user instance corresponding to a video stream request user exists in an edge computing end according to the user identifier and the user structure data queue; and the edge computing end is used for coding an image frame corresponding to the specified video stream acquisition equipment into a target video stream based on target user structure information in a user instance corresponding to the video stream request user, and feeding back the target video stream to the cloud server for distribution. Under the condition that the number of video stream request users is less than the number of video stream acquisition devices, the computing resource consumption is reduced, and the bandwidth occupation is reduced.

Description

Technical Field

[0001] This invention relates to the field of video stream processing and distribution, and in particular to a video stream distribution system, method, cloud server, and storage medium. Background Technology

[0002] With the widespread application of intelligent video surveillance systems, a large number of video stream acquisition devices have generated massive amounts of video data. This video data needs to undergo multiple processing stages, including decoding, intelligent algorithm analysis, business logic processing, and rendering, before it can be distributed to end users.

[0003] Existing technologies typically rely on centralized processing architectures, edge computing architectures, or cloud-edge collaborative processing architectures to process and distribute acquired video streams. Centralized processing architectures consist only of a cloud server, while edge computing and cloud-edge collaborative processing architectures each consist only of a cloud server and an edge computing terminal. Specifically, centralized processing architectures rely entirely on the cloud server to encode and process the entire video stream before distributing it to the end user. Edge computing and cloud-edge collaborative processing architectures, on the other hand, can encode and process the entire video stream at the edge computing terminal and send the processed video stream to the cloud server for subsequent distribution.

[0004] However, centralized processing architectures require the transmission of large amounts of unprocessed video data, resulting in high bandwidth consumption and latency. Secondly, while edge computing and cloud-edge collaborative processing architectures alleviate the burden on cloud servers and reduce transmission bandwidth, they still require the edge computing end to encode all video streams before pushing them to the cloud server, leading to severe consumption of computing resources, high bandwidth usage, and insufficient flexibility. Summary of the Invention

[0005] This invention provides a video stream distribution system, method, cloud server, and storage medium, which can encode only a portion of the video stream according to user needs when the number of users requesting the video stream is less than the number of video stream acquisition devices, thereby reducing computing resource consumption, reducing bandwidth usage, and making it more flexible.

[0006] In a first aspect, embodiments of the present invention provide a video stream distribution system, which includes multiple user interaction terminals, multiple edge computing terminals, and at least one cloud server;

[0007] The cloud server communicates with both the user interaction terminal and the edge computing terminal.

[0008] The user interaction terminal is used to receive user interaction commands from the video stream in response to user input.

[0009] The cloud server is used to parse user interaction commands and obtain user interaction information, which includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set.

[0010] Based on the user identifier and a pre-built user structure data queue, determine whether there is a user instance in the edge computing terminal that corresponds to the user requesting the video stream;

[0011] When it is determined that there is a user instance in the edge computing terminal corresponding to the user requesting the video stream, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information.

[0012] The edge computing terminal is used to encode the image frames corresponding to the video stream acquisition device into a target video stream based on the target user structure information in the user instance corresponding to the video stream requesting user, and send the target video stream to the user interaction terminal via the cloud server.

[0013] Secondly, embodiments of the present invention also provide a video stream distribution method, which is executed by a cloud server in a video stream distribution system of any embodiment of the present invention, the method comprising:

[0014] The user interaction commands sent by the user interaction terminal are parsed to obtain user interaction information; the user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set;

[0015] Based on the user identifier and a pre-built user structure data queue, determine whether there is a user instance in the edge computing terminal that corresponds to the user requesting the video stream;

[0016] When it is determined that there is a user instance in the edge computing terminal corresponding to the user requesting the video stream, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information.

[0017] The target video stream obtained from the edge computing terminal based on the target user's structure information is then sent to the user interaction terminal.

[0018] Thirdly, embodiments of the present invention also provide a cloud server, the cloud server comprising: at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the video stream distribution method provided in any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute and implement the video stream distribution method provided in any embodiment of the present invention.

[0020] The technical solution of this invention provides a video stream distribution system, which includes multiple user interaction terminals, multiple edge computing terminals, and at least one cloud server. The cloud server is used to determine whether a user instance corresponding to a user requesting a video stream exists in the edge computing terminals based on a user identifier and a pre-built user structure data queue. When a user instance corresponding to a user requesting a video stream is determined to exist in the edge computing terminals, the original user structure information corresponding to the user identifier in the user structure data queue is updated to target user structure information based on user interaction information. The edge computing terminals are used to encode image frames corresponding to the video stream acquisition device into a target video stream in the user instance corresponding to the user requesting the video stream, based on the target user structure information, and send the target video stream to the user interaction terminals via the cloud server. This solves the problem that when the number of users requesting video streams is less than the number of video stream acquisition devices, encoding and distributing all video streams through the edge computing terminals leads to severe consumption of computing resources, high bandwidth usage, and insufficient flexibility. This system allows encoding only a portion of the video streams according to user needs when the number of users requesting video streams is less than the number of video stream acquisition devices, thereby reducing computing resource consumption, bandwidth usage, and providing greater flexibility.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the structure of a video stream distribution system according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of a video stream distribution method provided in Embodiment 3 of the present invention;

[0025] Figure 3This is a flowchart of a preferred video stream distribution method provided by an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a cloud server provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a schematic diagram of a video stream distribution system according to Embodiment 1 of the present invention. This embodiment is applicable to situations where the collected video stream is distributed to the corresponding user.

[0031] like Figure 1 As shown, the video stream distribution system 11 disclosed in this embodiment includes: multiple user interaction terminals 101, multiple edge computing terminals 102 and at least one cloud server 103.

[0032] The cloud server 103 is communicatively connected to both the user interaction terminal 101 and the edge computing terminal 102. Specifically, the cloud server 103 includes a user structure data queue and an elastic streaming media service. The user structure data queue can be used to manage users requesting video streams and control the edge computing terminal 102 to appropriately process the video streams requested by each user. The elastic streaming media service can be used to distribute processed video streams to users requesting video streams. The user interaction terminal 101 can be used to receive user interaction commands input by users requesting video streams and display the requested video streams to the users. The edge computing terminal 102 can be used to receive raw video streams acquired by several video stream acquisition devices and perform preliminary processing such as decoding on each raw video stream. In addition, during the process of receiving and processing raw video streams, the edge computing terminal 102 can also be used to perform further processing such as encoding on some raw video streams according to user needs.

[0033] In one specific implementation, to establish a communication connection between the cloud server 103 and the user interaction terminal 101, a first short-connection HyperText Transfer Protocol (HTTP) service interface can be created in the cloud server 103. To establish a communication connection between the cloud server 103 and the edge computing terminal 102, a long-connection full-duplex communication protocol service interface can be established in the cloud server 103, and a second short-connection HTTP service interface can be established in the edge computing terminal 102.

[0034] In this embodiment, the user interaction terminal 101 is used to receive user interaction instructions input by the user requesting the video stream. The cloud server 103 is used to parse the user interaction instructions to obtain user interaction information; wherein, the user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, a stream processing field set, and a Web Real-Time Communication (WebRTC) protocol stream address used to uniquely identify the playback terminal; based on the user identifier and a pre-built user structure data queue, it is determined whether there is a user instance corresponding to the user requesting the video stream in the edge computing terminal 102; when it is determined that there is a user instance corresponding to the user requesting the video stream in the edge computing terminal 102, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information.

[0035] The user interaction command can only be used to transmit a video stream request command from a user requesting a single video stream. This means that if a user wants to browse the video streams from other video stream acquisition devices, they need to modify the user interaction command to switch to those devices. The user identifier uniquely identifies the user requesting the video stream; it can be the user's name, number, or other identifying information. The device identifier uniquely identifies the video stream acquisition device; it can be the device's Real-Time Streaming Protocol (RTSP) address, device serial number, or other identifying information. Video stream acquisition devices can be of various types, such as cameras and industrial cameras.

[0036] The stream processing field set can be understood as a set of fields used for further processing of the video stream after its initial processing by the edge computing terminal 102. The stream processing field set may include at least one of the browsing start / stop fields and rendering subfields. The playback terminal can be used to play the target video stream processed by the stream processing field set. The playback terminal may be located within the user interaction terminal 101.

[0037] In one specific implementation, the user interaction instructions of the video stream requesting user can be transmitted to the cloud server 103 via the user interaction terminal 101 using a first short-connection HTTP service interface. The input methods for the user interaction instructions can be various, such as keyboard, mouse, and gestures. Then, the cloud server 103 can query a pre-built user structure data queue based on the user identifier. If the query result indicates that there is no original user structure data corresponding to the user identifier in the user structure data queue, or only original user structure data with a blank instance identifier exists, it is determined that there is no user instance corresponding to the video stream requesting user in the edge computing terminal 102.

[0038] When the query result shows that there is original user structure data corresponding to the user identifier in the user structure data queue, and the instance identifier of the original user structure data is not blank, it is determined that there is a user instance corresponding to the video stream requesting user in the edge computing terminal 102.

[0039] When it is determined that there is a user instance in the edge computing terminal 102 corresponding to the user requesting the video stream, the original user structure information corresponding to the user identifier in the user structure data queue can be updated to the target user structure information based on the user interaction information, and the target user structure information can be sent to the edge computing terminal 102 through the long connection full-duplex communication protocol service interface and the second short connection HTTP service interface.

[0040] In this embodiment, the edge computing terminal 102 is used to encode the image frame corresponding to the video stream acquisition device into a target video stream based on the target user structure information in the user instance corresponding to the video stream requesting user, and send the target video stream to the user interaction terminal 101 via the cloud server 103.

[0041] In one specific implementation, each image frame corresponding to the video stream acquisition device can be obtained based on the device identifier in the target user structure information. Then, each image frame can be processed based on the stream processing field set in the target user structure information. Finally, each processed image frame can be encoded into a target video stream based on the WebRTC protocol stream address in the target user structure information.

[0042] Optionally, before and during image frame processing, the edge computing terminal 102 can also: create an image structure data queue for each connected video stream acquisition device through the edge computing box inside the edge computing terminal 102, and store each image structure data queue in a pre-created image storage container. Each image structure data queue uses a corresponding device identifier as a unique identifier. Then, the raw video stream acquired by the edge computing terminal 102 can be decoded into image frames using a software development kit (SDK) corresponding to the video stream acquisition device or a hardware decoding module of the edge computing box. Finally, a predefined intelligent algorithm can be used to process each image frame, obtaining the algorithm processing result corresponding to each image frame, and storing each image frame and its algorithm processing result in the corresponding image structure data queue. The aforementioned intelligent algorithm can be of various types, such as object detection algorithms and image enhancement algorithms. It is worth noting that if the amount of data in the image structure data queue is not less than a preset threshold, data is first taken from the head of the image structure data queue, and then new data is added to the tail of the image structure data queue; otherwise, new data is directly added to the tail of the image structure data queue. The preset threshold can be set according to user needs and historical experience; for example, the preset threshold can be set to 0.

[0043] The technical solution of this invention provides a video stream distribution system, which includes multiple user interaction terminals, multiple edge computing terminals, and at least one cloud server. The cloud server is communicatively connected to both the user interaction terminals and the edge computing terminals. The user interaction terminals are used to receive user interaction commands input by a user requesting a video stream. The cloud server is used to parse the user interaction commands to obtain user interaction information. This user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set. Based on the user identifier and a pre-built user structure data queue, it is determined whether a user instance corresponding to the user requesting the video stream exists in the edge computing terminals. When it is determined that a user instance corresponding to the user requesting the video stream exists in the edge computing terminals, based on the user interaction information... The system updates the original user structure information corresponding to the user identifier in the user structure data queue to the target user structure information. At the edge computing end, based on the target user structure information, it encodes the image frames corresponding to the video stream acquisition device into a target video stream within the user instance corresponding to the video stream requesting user. The target video stream is then sent to the user interaction terminal via the cloud server. This solves the problem of severe computational resource consumption, high bandwidth usage, and insufficient flexibility when the number of video stream requesting users is less than the number of video stream acquisition devices. Instead, it allows encoding only a portion of the video stream according to user needs when the number of video stream requesting users is less than the number of video stream acquisition devices, thereby reducing computational resource consumption, bandwidth usage, and providing greater flexibility.

[0044] Example 2

[0045] This embodiment is a further refinement of the above technical solution. The technical solution in this embodiment can be combined with the various optional solutions in one or more of the above embodiments.

[0046] like Figure 1 As shown, this embodiment discloses a video stream distribution system 11, including: multiple user interaction terminals 101, multiple edge computing terminals 102, and at least one cloud server 103. The cloud server 103 is communicatively connected to both the user interaction terminals 101 and the edge computing terminals 102.

[0047] In this embodiment, the cloud server 103 is specifically used to: determine whether there is original user structure data corresponding to the user identifier in the user structure data queue; if there is no original user structure data corresponding to the user identifier in the user structure data queue, then determine that there is no user instance corresponding to the video stream requesting user in the edge computing terminal 102; if there is original user structure data corresponding to the user identifier in the user structure data queue, then determine whether the instance identifier in the original user structure data is a blank value; if the instance identifier in the original user structure data is a blank value, then determine that there is no user instance corresponding to the video stream requesting user in the edge computing terminal 102; if the instance identifier field in the user structure data is not a blank value, then determine that there is a user instance corresponding to the video stream requesting user in the edge computing terminal 102.

[0048] Specifically, if it is determined that the original user structure data corresponding to the user identifier does not exist in the user structure data queue, it can be considered that the video stream requesting user is opening the video stream distribution system for the first time to browse video, and a new user instance can be created for the video stream requesting user. If it is determined that the original user structure data corresponding to the user identifier exists in the user structure data queue, but the instance identifier in the original user structure data is blank, it can be considered that the video stream requesting user is not opening the video stream distribution system for the first time to browse video, but has only previously sent an instruction to the video stream distribution system to stop browsing the video stream, and a new user instance can be created for the video stream requesting user.

[0049] Optionally, if it is determined that a user instance corresponding to the video stream requesting user exists in the edge computing terminal 102, the edge computing terminal 102 is specifically used to: read the stream processing field set in the target user structure information in the user instance corresponding to the video stream requesting user, and extract the browsing start / stop field in the stream processing field set; when the browsing start / stop field is "stop browsing video stream", release the user instance and send an instance identifier deletion command to the cloud server 103.

[0050] Specifically, when the browsing start / stop field indicates that video streaming is stopped, the thread identifier corresponding to the user requesting the video stream can be determined based on the instance identifier in the target user structure information. Then, the user thread corresponding to the aforementioned thread identifier can be closed to release the user instance. During the release of the user instance, an instance identifier deletion command can be sent to the cloud server 103, so that the cloud server 103 can modify the instance identifier in the target user structure information to a blank value.

[0051] The advantage of this setup is that by dynamically releasing user instances, it avoids occupying computing resources for encoding and other processing of useless video streams, thereby saving computing resources.

[0052] Optionally, if it is determined that a user instance corresponding to the video stream requesting user exists in the edge computing terminal 102, the edge computing terminal 102 is further configured to: extract the device identifier and the rendering sub-field set in the stream processing field set when the browsing start / stop field is set to not stop browsing the video stream; based on the device identifier, obtain the image structure data queue corresponding to the video stream acquisition device in the image storage container corresponding to the edge computing terminal 102; process each image frame in the image structure data queue according to the rendering sub-field set to obtain each processed image frame, and encode each processed image frame into the target video stream.

[0053] The rendering subfield set includes at least one of the following: rendering enable field, rendering object field, and rendering region field.

[0054] For example, assuming the rendering subfield set includes a rendering enable field, a rendering object field, and a rendering region field, then when the rendering enable field is set to prohibit rendering of image frames, each image frame in the image structure data queue can be directly transmitted to the user interaction terminal 101 in sequence. When the rendering enable field is set to allow rendering of image frames, the rendering object in each image frame is obtained from the algorithm processing results corresponding to each image frame, based on the rendering object field.

[0055] Then, based on the rendering region field, it can be determined whether the rendering object is within the rendering region of the corresponding image frame. Next, rendering objects within the rendering region are designated as renderable objects, while those outside are designated as non-renderable objects. Rendering is then performed only on the renderable objects to obtain the processed sequence of image frames. Each processed image frame includes both frames where the rendering object has been rendered and frames where it has not. Finally, based on the WebRTC protocol stream address in the target user structure data, video stream encoding is performed on each processed image frame, and the encoded video stream is sent to the cloud server 103.

[0056] The advantage of this setup is that by rendering the video stream through the set of rendering subfields in the user interaction commands, rendering of video streams that do not need to be rendered can be avoided, thereby saving computing resources.

[0057] Optionally, the cloud server 103 is further configured to send an instance creation instruction to the edge computing terminal 102 when it is determined that there is no user instance corresponding to the video stream requesting user in the edge computing terminal 102; the edge computing terminal 102 is further configured to create a user instance corresponding to the video stream requesting user according to the instance creation instruction, and send the instance identifier of the user instance to the cloud server 103; the cloud server 103 is further configured to generate current user structure data corresponding to the user according to the instance identifier and user interaction information when it receives the instance identifier, and send the current user structure data to the edge computing terminal 102; the edge computing terminal 102 is configured to encode the image frame corresponding to the video stream acquisition device into the current video stream in the user instance corresponding to the video stream requesting user based on the current user structure information, and send the current video stream to the user interaction terminal 101 via the cloud server 103.

[0058] Specifically, upon receiving an instance identifier, the cloud server 103 can directly generate the current user structure data corresponding to the user based on the instance identifier and user interaction information. Alternatively, after processing the user identifier and WebRTC protocol stream address in the user interaction information into updated user identifiers and updated WebRTC protocol stream addresses according to preset rules, the current user structure data corresponding to the user can be generated based on the instance identifier and the updated user interaction information containing the updated user identifier and updated WebRTC protocol stream address. Taking the user identifier as the name of the user requesting the video stream as an example, the method for processing the user identifier into an updated user identifier can be: performing a custom encoding mapping on the user identifier to obtain the updated user identifier. Furthermore, the method for processing the WebRTC protocol stream address into an updated WebRTC protocol stream address can be: using the updated user identifier as the last field of the updated WebRTC protocol stream address.

[0059] After generating the current user structure data corresponding to the user, the current user structure data can be stored in the user structure data queue through the cloud server 103 and sent to the edge computing terminal 102.

[0060] After the current user structure data is sent to the edge computing terminal 102 via the cloud server 103, the device identifier and rendering subfield set can be extracted from the current user structure data. Then, based on the device identifier, the image structure data queue corresponding to the video stream acquisition device can be obtained from the image storage container corresponding to the edge computing terminal 102. Finally, each image frame in the image structure data queue can be processed according to the rendering subfield set to obtain processed image frames, and the processed image frames are encoded into the current video stream.

[0061] Optionally, the cloud server 103 is specifically used to: when it is determined that there is no user instance corresponding to the video stream requesting user in the edge computing terminal 102, generate an instance creation instruction based on the user identifier, and send the instance creation instruction to the edge computing terminal 102; the edge computing terminal 102 is specifically used to: read the user identifier in the instance creation instruction, create a user instance based on the user identifier, and generate an instance identifier corresponding to the user instance; wherein, one user instance corresponds to one user thread.

[0062] Specifically, when it is determined that no user instance corresponding to the user requesting the video stream exists in the edge computing terminal 102, an instance creation instruction is generated based on the user identifier and sent to the edge computing terminal 102 via a second short-connection HTTP service interface. Then, the edge computing terminal 102 can create a user thread based on the user identifier and generate a thread identifier corresponding to the user thread. The user thread is a sub-thread under the main thread corresponding to the edge computing terminal 102, and can be used to handle all user interaction instructions for its corresponding user. Finally, a user instance corresponding to the user thread can be created based on the thread identifier of the user thread, and the thread identifier is used as the instance identifier of the corresponding user instance.

[0063] The advantage of this setup is that by dynamically creating user instances, computing resources can be dynamically allocated according to the actual needs of users, enabling the video streaming system to flexibly cope with different numbers and needs of users.

[0064] The technical solution in this embodiment avoids the resource waste of encoding and streaming all video streams by encoding and streaming only the specific video stream requested by the user, reducing bandwidth consumption, lowering transmission latency, and meeting the requirements of low latency and high efficiency. Secondly, by dynamically creating and releasing user instances, and dynamically allocating and releasing computing resources according to the user's actual needs, the video stream distribution system can flexibly cope with different numbers and needs of users, saving computing resources while ensuring user satisfaction. Finally, by rendering the video stream through the rendering subfield set in the user interaction command, rendering unnecessary video streams can be avoided, thereby saving computing resources.

[0065] Example 3

[0066] Figure 2 This is a flowchart of a video stream distribution method according to Embodiment 3 of the present invention. This embodiment is a further refinement of the above technical solution. The technical solution in this embodiment can be combined with the various optional solutions in one or more of the above embodiments.

[0067] like Figure 2As shown, this embodiment discloses a video stream distribution method, which is executed by a cloud server in the video stream distribution system of any embodiment of the present invention. The method includes:

[0068] S110. Parse the user interaction instructions sent by the user interaction terminal to obtain user interaction information; wherein, the user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set.

[0069] In this embodiment, the user interaction information may also include a WebRTC protocol stream address used to uniquely identify the playback terminal.

[0070] S120. Based on the user identifier and the pre-built user structure data queue, determine whether there is a user instance in the edge computing terminal corresponding to the user requesting the video stream.

[0071] Specifically, this step involves determining whether original user structure data corresponding to the user identifier exists in the user structure data queue. If no original user structure data corresponding to the user identifier exists in the user structure data queue, it is determined that no user instance corresponding to the video stream requesting user exists in the edge computing terminal. If original user structure data corresponding to the user identifier exists in the user structure data queue, it is determined whether the instance identifier in the original user structure data is blank. If the instance identifier in the original user structure data is blank, it is determined that no user instance corresponding to the video stream requesting user exists in the edge computing terminal. If the instance identifier field in the user structure data is not blank, it is determined that a user instance corresponding to the video stream requesting user exists in the edge computing terminal.

[0072] S130. When it is determined that there is a user instance in the edge computing terminal corresponding to the user requesting the video stream, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information.

[0073] S140. Obtain the target video stream from the edge computing terminal based on the target user structure information, and send the target video stream to the user interaction terminal.

[0074] Optionally, the method further includes: when it is determined that there is no user instance corresponding to the user requesting the video stream in the edge computing terminal, sending an instance creation instruction to the edge computing terminal; upon receiving an instance identifier fed back by the edge computing terminal based on the instance creation instruction, generating current user structure data corresponding to the user according to the instance identifier and user interaction information, and sending the current user structure data to the edge computing terminal; obtaining the current video stream obtained by the edge computing terminal based on the current user structure information in the user instance corresponding to the user requesting the video stream, and sending the current video stream to the user interaction terminal.

[0075] Furthermore, sending an instance creation instruction to the edge computing terminal may include: when it is determined that there is no user instance corresponding to the user requesting the video stream in the edge computing terminal, generating an instance creation instruction based on the user identifier, and sending the instance creation instruction to the edge computing terminal.

[0076] The technical solution of this embodiment obtains user interaction information by parsing user interaction commands sent by the user interaction terminal. This user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set. Based on the user identifier and a pre-built user structure data queue, it is determined whether a user instance corresponding to the video stream requesting user exists in the edge computing terminal. When it is determined that a user instance corresponding to the video stream requesting user exists in the edge computing terminal, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information. The technical means of obtaining the target video stream obtained by the edge computing terminal based on the target user structure information and sending the target video stream to the user interaction terminal solves the problem of severe computational resource consumption, high bandwidth usage, and insufficient flexibility when the number of video stream requesting users is less than the number of video stream acquisition devices. This allows for encoding only a portion of the video stream according to user needs when the number of video stream requesting users is less than the number of video stream acquisition devices, thereby reducing computational resource consumption, bandwidth usage, and providing greater flexibility.

[0077] To illustrate the video stream distribution method and its effects in this invention in detail, a specific embodiment is described below: Figure 3 As shown, Step 1: Create an image storage container at the edge computing end and a user structure data queue on the cloud server. The image storage container stores the image structure data queue corresponding to each connected video stream acquisition device. The user structure data queue stores the user structure data corresponding to each historical logged-in user. Step 2: Create a first short-connection HTTP service interface and a long-connection full-duplex communication protocol service interface on the cloud server, and a second short-connection HTTP service interface on the edge computing end to establish communication connections between the cloud server, the user interaction end, and the edge computing end. Step 3: Decode the video streams acquired by each connected video stream acquisition device through the edge computing end to obtain a large number of image frames corresponding to each connected video stream acquisition device. Then, use an intelligent algorithm to process each image frame to obtain the algorithm processing result corresponding to each image frame. Finally, store the image frames and their algorithm processing results in the corresponding image structure data queue according to the acquisition order. The image structure data queue is a first-in, first-out queue.

[0078] Step 4: Receive user interaction commands input by the user in response to the video stream request through the user interaction terminal, and send the user interaction commands to the cloud server. The playback terminal of the user interaction terminal includes a signaling interaction module and an audio / video data decoding and display module. Specifically, in response to the user interaction commands input by the user in response to the video stream request on the playback terminal, the signaling interaction module can be invoked to request the first short-connection HTTP service interface of the cloud server, and the user interaction commands can be sent to the cloud server. It is worth noting that since Step 3 is a continuously executed step, Step 4 can be triggered during the execution of Step 3.

[0079] Step 5: Parse the user interaction commands through the cloud server to obtain user interaction information. This information includes a user identifier, a device identifier for the user-specified video stream acquisition device, a stream processing field set, and a WebRTC protocol stream address to uniquely identify the playback terminal. Step 6: Based on the user identifier and the user structure data queue, determine if the video stream requesting user is using the video stream distribution system for the first time. If it is, a user instance is created for the user, and this instance is used to process all user interaction commands for that user, thereby allocating the required video stream to the user. Step 7: If the user is not using the video stream distribution system for the first time (i.e., the original user structure data corresponding to the user exists in the user structure data queue), determine whether a new user instance needs to be created for the user. When it is determined that a new user instance needs to be created for the user requesting the video stream, that is, when the instance identifier in the original user structure data is blank, a new user instance is created for the user requesting the video stream. The user instance is then invoked to process all user interaction instructions of the user requesting the video stream, thereby allocating the required video stream to the user requesting the video stream.

[0080] Step 8: When it is determined that no new user instance needs to be created for the user requesting the video stream (i.e., the instance identifier in the original user structure data is not blank), check whether the "Browse Start / Stop" field in the stream processing field set indicates stopping video streaming. If the "Browse Start / Stop" field indicates stopping video streaming, release the user instance and send an instance identifier deletion command to the cloud server. If the "Browse Start / Stop" field indicates not stopping video streaming, run the user instance corresponding to the user requesting the video stream, process user interaction commands, and push the required video stream to the user requesting the video stream. Optionally, after pushing the required video stream to the user requesting the video stream, the pushed video stream can be received and decoded and played through the audio and video data decoding and display module in the user interaction terminal.

[0081] The advantage of this setup is that by acquiring user interaction information for each video stream requesting user, user instances can be created for first-time login users and non-first-time login users who have previously stopped browsing video streams, and by running user instances to process and distribute video streams for video stream requesting users, it can reduce computing resource consumption, reduce bandwidth usage, and be more flexible when the number of video stream requesting users is less than the number of video stream acquisition devices.

[0082] Example 4

[0083] Figure 4 A schematic diagram of the structure of a cloud server that can be used to implement embodiments of the present invention is shown. Figure 4 As shown, the cloud server includes at least one processor 21 and memory, such as read-only memory (ROM) 22 and random access memory (RAM) 23, communicatively connected to the at least one processor 21. The memory stores computer programs executable by the at least one processor. The processor 21 can perform various appropriate actions and processes based on the computer programs stored in the ROM 22 or loaded from storage unit 28 into the RAM 23. The RAM 23 can also store various programs and data required for the operation of the cloud server. The processor 21, ROM 22, and RAM 23 are interconnected via a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0084] Multiple components in the cloud server are connected to I / O interface 25, including: input unit 26, such as keyboard, mouse, etc.; output unit 27, such as various types of monitors, speakers, etc.; storage unit 28, such as disk, optical disk, etc.; and communication unit 29, such as network card, modem, wireless communication transceiver, etc. Communication unit 29 allows the cloud server to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0085] Processor 21 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 21 performs the various methods and processes described above, such as video streaming distribution methods.

[0086] In some embodiments, the video stream distribution method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on a cloud server via ROM 22 and / or communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the video stream distribution method described above may be performed. Alternatively, in other embodiments, processor 21 may be configured to perform the video stream distribution method by any other suitable means (e.g., by means of firmware).

[0087] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0090] To provide interaction with the user, the systems and techniques described herein can be implemented on a cloud server 103, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the cloud server 103. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0091] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0092] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0093] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0094] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video stream distribution system, characterized by, The video stream distribution system includes multiple user interaction terminals, multiple edge computing terminals, and at least one cloud server; The cloud server is communicatively connected to both the user interaction terminal and the edge computing terminal. The user interaction terminal is used to receive user interaction commands input by the user in response to video stream requests; The cloud server is used to parse the user interaction instructions to obtain user interaction information; wherein, the user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set; Based on the user identifier and the pre-built user structure data queue, determine whether there is a user instance in the edge computing terminal corresponding to the user requesting the video stream; When it is determined that there is a user instance in the edge computing terminal corresponding to the user requesting the video stream, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information. The edge computing terminal is used to encode the image frame corresponding to the video stream acquisition device into a target video stream based on the target user structure information in the user instance corresponding to the video stream requesting user, and send the target video stream to the user interaction terminal via the cloud server.

2. The system of claim 1, wherein, The cloud server is specifically used for: Determine whether the user structure data queue contains original user structure data corresponding to the user identifier; If the original user structure data corresponding to the user identifier does not exist in the user structure data queue, then it is determined that there is no user instance corresponding to the video stream requesting user in the edge computing terminal. If the user structure data queue contains original user structure data corresponding to the user identifier, then determine whether the instance identifier in the original user structure data is a blank value. If the instance identifier in the original user structure data is blank, it is determined that there is no user instance corresponding to the video stream requesting user in the edge computing terminal; If the instance identifier field in the user structure data is not blank, then it is determined that there is a user instance in the edge computing terminal corresponding to the user requesting the video stream.

3. The system of claim 1, wherein, The edge computing terminal is specifically used for: In the user instance corresponding to the video stream requesting user, read the stream processing field set in the target user structure information, and extract the browsing start / stop field from the stream processing field set; When the browsing start / stop field indicates that the video stream is stopped, the user instance is released, and an instance identifier deletion command is sent to the cloud server.

4. The system of claim 3, wherein, The edge computing terminal is also used for: When the browsing start / stop field is set to not stop browsing the video stream, extract the device identifier and the rendering subfield set from the stream processing field set; Based on the device identifier, the image structure data queue corresponding to the video stream acquisition device is obtained from the image storage container corresponding to the edge computing terminal; Based on the rendering subfield set, each image frame in the image structure data queue is processed to obtain processed image frames, and the processed image frames are encoded into a target video stream.

5. The system according to claim 4, characterized in that, The rendering subfield set includes at least one of the following: rendering enable field, rendering object field, and rendering region field.

6. The system according to claim 1, characterized in that, The cloud server is also used to send an instance creation instruction to the edge computing terminal when it is determined that there is no user instance corresponding to the user requesting the video stream in the edge computing terminal; The edge computing terminal is also used to create a user instance corresponding to the video stream requesting user according to the instance creation instruction, and send the instance identifier of the user instance to the cloud server; The cloud server is also configured to, upon receiving the instance identifier, generate current user structure data corresponding to the user based on the instance identifier and the user interaction information, and send the current user structure data to the edge computing terminal; The edge computing terminal is used to encode the image frame corresponding to the video stream acquisition device into a current video stream based on the current user structure information in the user instance corresponding to the video stream requesting user, and send the current video stream to the user interaction terminal via the cloud server.

7. The system according to claim 1, characterized in that, The cloud server is specifically used to: when it is determined that there is no user instance corresponding to the video stream requesting user in the edge computing terminal, generate an instance creation instruction based on the user identifier, and send the instance creation instruction to the edge computing terminal; The edge computing terminal is specifically used for: reading the user identifier in the instance creation instruction, creating a user instance based on the user identifier, and generating an instance identifier corresponding to the user instance; wherein, one user instance corresponds to one user thread.

8. A method of video stream distribution, characterized by, The method is performed by a cloud server in the video stream distribution system as described in any one of claims 1 to 7, the method comprising: The user interaction instructions sent by the user interaction terminal are parsed to obtain user interaction information; wherein, the user interaction information includes a user identifier, a device identifier of the video stream acquisition device specified by the user, and a stream processing field set; Based on the user identifier and the pre-built user structure data queue, determine whether there is a user instance in the edge computing terminal corresponding to the user requesting the video stream; When it is determined that there is a user instance in the edge computing terminal corresponding to the user requesting the video stream, the original user structure information corresponding to the user identifier in the user structure data queue is updated to the target user structure information based on the user interaction information. The edge computing terminal obtains the target video stream based on the target user structure information and sends the target video stream to the user interaction terminal.

9. A cloud server, characterized by The cloud server includes: At least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video stream distribution method of claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the video stream distribution method of claim 8.

Images