Video stream frame cutting method and related device

Abstract

The invention discloses a video stream frame intercepting method and a related device, an online list can comprise live broadcast information of online live broadcast rooms, so that after the online list is acquired, whether a target live broadcast room meeting a frame intercepting time condition exists in the online live broadcast rooms or not can be determined according to the online list, and if yes, the video stream is intercepted. In this way, the frame cutting task corresponding to the target live broadcast room can be generated, and the cut frames of the live broadcast video stream of the online live broadcast room can be maintained in a unified manner. The frame cutting task is executed on the live broadcast video stream of the target live broadcast room through the frame cutting module, the frame cutting module comprises a plurality of frame cutting units, and the frame cutting units are used for executing the frame cutting task, so that the frame cutting task can be executed through any frame cutting unit in the frame cutting units, and the frame cutting task can be executed even if more frame cutting tasks are caused by more online live broadcast rooms. Task accumulation caused by a single frame interception unit in the execution process of the frame interception task is avoided, and the task frame interception efficiency can be improved under the condition of maintaining a plurality of live broadcasting rooms.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a video stream frame capture method and related devices. Background Art

[0002] Frame capture refers to extracting a static image at a certain moment from a video stream, similar to taking a photo. Frame capture is often used to extract key frames from a video stream. In the process of capturing frames of a video stream, a frame capture task can be generated for a specific video stream and executed to generate a frame capture image corresponding to the video stream. However, currently, a specific video stream is often captured by a specific frame capture device, which easily causes a large load on the frame capture device. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a video stream frame capture method and related devices, which can improve the task frame capture efficiency while maintaining multiple live broadcast rooms by executing the frame capture task in parallel through multiple frame capture units in the frame capture module.

[0004] The embodiments of the present application disclose the following technical solutions:

[0005] On the one hand, the present application provides a video stream frame capture method, the method comprising:

[0006] Get an online list including live broadcast information of online live broadcast rooms;

[0007] When it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast rooms, a frame capture task corresponding to the target live broadcast room is generated;

[0008] The frame capture task is performed on the live video stream of the target live broadcast room by a frame capture module including multiple frame capture units to capture the frame capture image corresponding to the target live broadcast room, and any one of the multiple frame capture units is used to perform the frame capture task.

[0009] On the other hand, the present application provides a video stream frame capture device, the device comprising:

[0010] An online list acquisition unit, used to acquire an online list including live broadcast information of an online live broadcast room;

[0011] A frame capture task generating unit, configured to generate a frame capture task corresponding to the target live broadcast room when it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast rooms;

[0012] A frame capture task execution unit is used to execute the frame capture task on the live video stream of the target live broadcast room through a frame capture module including multiple frame capture units to capture the frame capture image corresponding to the target live broadcast room, and any one of the multiple frame capture units is used to execute the frame capture task.

[0013] In another aspect, the present application provides a computer device, the device comprising a processor and a memory:

[0014] The memory is used to store a computer program and transmit the computer program to the processor;

[0015] The processor is used to execute the video stream frame capture method described in the above aspect according to the instructions in the computer program.

[0016] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the video stream frame capture method described in the above aspect.

[0017] On the other hand, an embodiment of the present application provides a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the video stream frame capture method.

[0018] It can be seen from the above technical solution that the online list can include the live broadcast information of the online live broadcast room. In this way, after obtaining the online list, it can be determined whether there is a target live broadcast room that meets the frame capture time condition in the online live broadcast room according to the online list. If it exists, it means that the target live broadcast room needs to be frame captured. At this time, a frame capture task corresponding to the target live broadcast room can be generated, so that the frame capture of the live video stream of the online live broadcast room can be uniformly maintained. Then, the frame capture task is executed on the live video stream of the target live broadcast room through the frame capture module. The frame capture module includes multiple frame capture units, and the frame capture unit is used to execute the frame capture task. In this way, the frame capture task can be executed by any frame capture unit in the frame capture unit. Even if there are many online live broadcast rooms resulting in many frame capture tasks, the frame capture task is executed in parallel by multiple frame capture units in the frame capture module, avoiding the task accumulation caused by a single frame capture unit during the execution of the frame capture task, and improving the task frame capture efficiency when maintaining multiple live broadcast rooms. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of an application scenario of a video stream frame capture method provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a video stream frame capture method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the architecture of a live broadcast system provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the architecture of another live broadcast system provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of an online list provided in an embodiment of the present application;

[0025] Figure 6 A process diagram of a heartbeat mechanism provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of a frame-cut image provided in an embodiment of the present application;

[0027] Figure 8 A structural block diagram of a video stream frame capture device provided in an embodiment of the present application;

[0028] Fig. 9 A structural diagram of a terminal device provided in an embodiment of the present application;

[0029] Fig.10 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0031] In the process of capturing frames of a video stream, a frame capturing task can be generated for a specific video stream and executed to generate a frame capturing image corresponding to the video stream. However, currently, a specific video stream is often captured by a specific frame capturing device, which easily causes a heavy load on the frame capturing device.

[0032] In order to solve the above technical problems, the live broadcast information of the online live broadcast room can be reflected by setting an online list. When it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast room, a frame capture task corresponding to the target live broadcast room is generated, so that the frame capture of the live video stream of the online live broadcast room can be uniformly maintained. By executing the frame capture task in parallel by multiple frame capture units in the frame capture module, the efficiency of the task frame capture can be improved when maintaining multiple live broadcast rooms.

[0033] The video stream frame capture method provided in the embodiment of the present application can be implemented by a computer device, which can be a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The server can be connected to a terminal device, and the terminal device includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0034] The collection and processing of relevant data in this application (such as live video streams, live broadcast information, etc.) should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0035] In order to facilitate understanding of the technical solution provided by the present application, a video stream frame capture method provided by an embodiment of the present application will be introduced below in combination with an actual application scenario.

[0036] Figure 1 A schematic diagram of an application scenario of a video stream frame capture method provided in an embodiment of the present application is shown, the scenario includes a server 10, a storage node 20 and a frame capture module 30, wherein the server 10 is used as the aforementioned computer device to execute the video stream frame capture method, the storage node 20 can be used to store an online list, and the server 10 can obtain the online list from the storage node 20.

[0037] The online list may include the live broadcast information of the online live broadcast room. In this way, after obtaining the online list, the server 10 can determine whether there is a target live broadcast room in the online live broadcast room that meets the frame capture time condition based on the online list. If so, it means that the target live broadcast room needs to be captured. At this time, a frame capture task corresponding to the target live broadcast room can be generated, so that the frame capture of the live video stream of the online live broadcast room can be uniformly maintained.

[0038] Then the server 10 can execute the frame capture task on the live video stream of the target live broadcast room through the frame capture module 30. The frame capture module 30 includes multiple frame capture units, and the frame capture units are used to execute the frame capture task. In this way, the frame capture task can be executed by any frame capture unit. Even when there are many online live broadcast rooms resulting in a large number of frame capture tasks, the frame capture task can be executed in parallel by multiple frame capture units in the frame capture module 30, thereby avoiding task accumulation caused by a single frame capture unit during the execution of the frame capture task, and improving the task capture efficiency when maintaining multiple live broadcast rooms.

[0039] Figure 2A flowchart of a video stream frame capture method provided in an embodiment of the present application. In this embodiment, a server is used as the aforementioned computer device for description. The server includes a scheduling cluster. The following is an example of the scheduling cluster executing the video stream frame capture method. The video stream frame capture method may include:

[0040] S101, obtaining an online list including live broadcast information of online live broadcast rooms.

[0041] Live broadcast is a broadcasting method for radio and television programs. In this broadcasting method, post-production (haobc) synthesis and broadcasting are carried out simultaneously. Live broadcast can realize real-time online communication, which allows users to timely transmit video, text and audio streams through the network to enrich the form of online communication. The live broadcast room is a network live broadcast program that can be broadcast live through the live broadcast room. The live broadcast room is a platform for content providers and viewers to communicate. The content provider forms a live video stream through recording. The audience watches the live video stream through the live broadcast room and receives corresponding information. During the live broadcast, the audience can interact with the content provider through the live broadcast room, such as commenting, sending gifts, shopping, etc. The live broadcast room can have a live broadcast identifier, which is used to uniquely identify the live broadcast room. The live broadcast room is usually associated with the content provider.

[0042] In the embodiment of the present application, the live video stream of the live broadcast room can be cut to obtain the cut frame image, which can be used in the fields of video editing, analysis and processing, and can be used as the key frame of the live video stream for making thumbnails, image analysis or video analysis, etc. Among them, the online live broadcast room refers to the live broadcast room that is currently broadcasting. By cutting the live video stream of the online live broadcast room, the cut frame image of the online live broadcast room can be obtained in real time, and the real-time monitoring and analysis of the live video stream of the online live broadcast room can be realized. The live video stream is converted into an image sequence that is easy to understand manually, which helps auditors quickly discover and deal with live content suspected of violating regulations without spending a lot of manpower and material resources focusing on the live video stream.

[0043] During live broadcast, the terminal device can obtain the live video stream and live broadcast information of the live broadcast room. The live video stream is obtained through recording. The live broadcast information may include live broadcast identification, interactive information, broadcast start information, broadcast end information, etc. Interactive information may include comment information, gift information, shopping information, audience change information, etc. The terminal device sends the live video stream and live broadcast information to the network through a network connection, so as to interact with users of other terminal devices.

[0044] refer to Figure 3As shown, it is a schematic diagram of the architecture of a live broadcast system provided by an embodiment of the present application. The terminal device 110 can send live broadcast information to the live broadcast server 120, and the live broadcast server 120 sends the live broadcast information to the live broadcast access layer 130. The live broadcast access layer 130 is used to support the live broadcast room to access the network, so that the live broadcast room can be managed based on the live broadcast information in the network. The terminal device 110 can send a live video stream to the live content distribution network (Content Delivery Network, CDN) 140. The live content distribution network 140 includes multiple CDN nodes. Data can be transmitted between multiple CDN nodes, so that the live video stream is pushed to the edge node close to the user among the multiple CDN nodes, so that the user can obtain resources nearby, thereby increasing the user's access speed and access stability.

[0045] In the embodiment of the present application, an online live broadcast room can be determined based on the start-up information and the end-of-broadcast information of the live broadcast room. The online live broadcast room is the live broadcast room that is currently generating a live video stream. Then, an online list is generated based on the live broadcast information of the online live broadcast room. The online list can reflect the list of live broadcast rooms that are currently broadcasting. The live broadcast room list may include at least one live broadcast identification of the live broadcast room, and the live broadcast identification is used to uniquely identify the live broadcast room to which it corresponds. Among them, the start-up information of the live broadcast room indicates the time when the live broadcast room starts broadcasting, and the end-of-broadcast information of the live broadcast room indicates the time when the live broadcast room ends broadcasting. If the live broadcast room only has the start-up information but no end-of-broadcast information or the end-of-broadcast information is empty, the live broadcast room is determined to be an online live broadcast room. If the live broadcast room has the start-up information and the end-of-broadcast information is not empty, the live broadcast room is determined to be a non-online live broadcast room.

[0046] The module for generating the online list can be a scheduling cluster or a list generation module for generating lists. The scheduling cluster is a module in the server for data processing. The module for generating the online list can obtain the start and end broadcast information of the live broadcast room from the live broadcast server through the live broadcast access layer, and thus determine the online live broadcast room according to the start broadcast information. Figure 4 As shown, it is a schematic diagram of the architecture of another live broadcast system provided by an embodiment of the present application, the scheduling cluster 150 is connected to the live broadcast access layer 130, so as to obtain the start and end broadcast information of the live broadcast room through the live broadcast access layer 130. In addition, the scheduling cluster 150 can generate an online list and store the online list in the storage node 20. The storage node 20 and the scheduling cluster 150 can be set in the same server or in different servers.

[0047] Online lists can be stored in multiple storage nodes. The online lists in multiple storage nodes have the same content, realizing disaster recovery storage of multiple copies of online lists. Disaster recovery refers to the ability to maintain high availability, reliability and stability through the use of technical means and measures when a system or service is subject to unavoidable risks such as possible natural disasters, man-made damage, hardware failure, etc. The purpose of disaster recovery is to ensure the continuity of the system or service to ensure the normal operation of the business and the security of the data. Backup storage is one of the means of disaster recovery.

[0048] Multiple storage nodes can form a distributed system. A distributed system is a computing model that distributes the tasks and data of a computer system to multiple computer nodes for processing and storage. In a distributed system, each node communicates and collaborates with each other through the network to jointly complete tasks and process data. Distributed systems have the advantages of high availability, scalability, and high fault tolerance, which can improve the performance and reliability of the system and are suitable for processing large-scale data and high concurrent access scenarios. Therefore, multiple online lists in multiple storage nodes can be accessed at the same time, so as to simultaneously process processing requests related to online lists, improve the system's reading performance, improve access efficiency, and reduce the load pressure of a single storage node.

[0049] In addition, since the online list is stored in multiple storage nodes, and the online lists stored in each storage node have the same content, taking the online list of one storage node as an example, the online lists of other storage nodes can be used as copies or backups of the online list in the storage node, thereby realizing multi-copy storage of the online list. Multi-copy storage refers to a data replication method in which multiple copies of data are stored on different nodes in a distributed system. Multi-copy storage can improve the availability, fault tolerance and performance of the system. When a storage node fails or the network is interrupted, the copies of other storage nodes can continue to provide services to ensure the normal operation of the system. This multi-copy storage can also be called a backup mechanism. The backup mechanism refers to backing up the system data and applications to another one or more storage nodes in order to ensure the high availability of the system, so that when the main node fails, it can quickly switch to the backup node, thereby ensuring the continuity and availability of the system. The failure of a storage node may be caused by a variety of reasons such as hardware failure, software error, network failure, etc. If multi-copy storage or backup mechanism is not set, the failure of such a storage node may cause some or all services of the entire system to be unavailable.

[0050] The online list can be obtained by the scheduling cluster. When the online list is stored in multiple storage nodes, obtaining the online list can specifically be to obtain the online list stored in the target node among the multiple storage nodes, including the live broadcast information of the online live broadcast room, wherein the target node can be the main node among the multiple storage nodes, or can be the storage node among the multiple storage nodes corresponding to the scheduling cluster.

[0051] The online list can be stored in the form of key-value pairs (KV). Key-value pair storage is a lightweight data storage method that can be quickly read and written. Among them, the online list can include a key-value pair consisting of the live broadcast identifier of the online live broadcast room and other live broadcast information other than the live broadcast identifier. The live broadcast identifier is used as the key, and the other live broadcast information other than the live broadcast identifier is used as the value. Each live broadcast identifier is used to uniquely identify a live broadcast room, and the other live broadcast information corresponding to it is the live broadcast information of the live broadcast room, thereby forming a live broadcast list that can be used to manage online live broadcast rooms. Reference Figure 5 As shown, it is a schematic diagram of an online list provided in an embodiment of the present application. The live broadcast identifier may include live broadcast identifiers 1 to n, and each live broadcast identifier corresponds to other live broadcast information.

[0052] After the online list is generated, the online list can also be sent to the terminal device so that the terminal device can display the online list. The terminal device that displays the online list can be a terminal device with a live broadcast platform, so that the terminal device has a device that provides a live broadcast function. The online list can be displayed in the live broadcast platform of the terminal device, for example, on the homepage or classification page of the live broadcast platform, so that the user can understand the current online live broadcast room. The scheduling cluster can send the online list to the terminal device directly, or send the online list to the terminal device through other modules or live broadcast servers.

[0053] In an embodiment of the present application, after the online list is generated, the online list can also be updated. This is because after the live broadcast room is started, data may be lost due to packet loss or server abnormalities during data transmission. In this way, the live broadcast room cannot be successfully added to the online list, and the live broadcast room cannot be captured. By updating the online list, it can be ensured that the online live broadcast room can exist in the online list as much as possible, thereby realizing the recovery of lost data.

[0054] The update of the online list can be performed by a module that generates an online list, such as a scheduling cluster or a list generation module. Specifically, the scheduling cluster can obtain the heartbeat packet corresponding to the live broadcast room to be detected through the live broadcast access layer. If the live broadcast room to be detected does not exist in the online list, the live broadcast room to be detected can be added to the online list, wherein the heartbeat packet may include at least one of the online identification and interactive information corresponding to the live broadcast room to be detected. The live broadcast server can generate a heartbeat packet based on the acquired live broadcast information of the live broadcast room, wherein the interactive information may include comment information, gift sending information, shopping information, audience change information, etc. The online identification is used to indicate that the live broadcast room is online, that is, the interactive information and other actions in the live broadcast service process can be used as a heartbeat packet, thereby ensuring that the lost data in abnormal circumstances can be smoothly restored and the comprehensiveness of the frame capture is ensured.

[0055] That is to say, in the embodiment of the present application, the online list can be maintained by the heartbeat mechanism, wherein the heartbeat mechanism is a mechanism for regularly sending heartbeat packets in order to maintain the connection status between nodes, which can detect the connection status between nodes and prevent the nodes from losing connection or downtime, resulting in failure of the entire system. The heartbeat packet here is used to monitor the connection status between the two nodes of the scheduling cluster and the live access layer. Of course, the online information and interactive information corresponding to the live broadcast room to be detected can come from the live broadcast server, so the connection status between the three nodes of the live broadcast server, the scheduling cluster and the live broadcast access layer can be monitored through the heartbeat packet.

[0056] refer to Figure 6 As shown, it is a process diagram of a heartbeat mechanism provided in an embodiment of the present application. The live broadcast server 120 can send a heartbeat packet to the live broadcast access layer 130 so that the scheduling cluster 150 can obtain the heartbeat packet through the live broadcast access layer 130, thereby updating the online list according to the live broadcast room to be detected corresponding to the heartbeat packet.

[0057] S102, when it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast rooms, a frame capture task corresponding to the target live broadcast room is generated.

[0058] In the traditional frame-cutting process, a single frame-cutting machine is usually used to perform the frame-cutting task for a specific video stream at a fixed time. It adopts the form of a message queue (MQ). The task producer of the frame-cutting machine first generates the frame-cutting task into the message queue, and then the task consumer in the frame-cutting machine obtains the frame-cutting task from the message queue and executes the frame-cutting task, realizing the decoupling between the task producer and the task consumer. After the frame-cutting task is dispatched, a new frame-cutting task is generated by self-looping to realize continuous frame-cutting. Message queue is a technology for asynchronous communication in a distributed system. Message queue can improve the scalability, availability and reliability of the system, and can handle scenarios such as high concurrency, large data volume, and asynchronous communication. However, in this frame-cutting mode, each frame-cutting machine only cuts frames for a specific video stream, which results in serial execution between each frame-cutting task and the next frame-cutting task, low task scheduling efficiency, and low frame-cutting efficiency. In addition, fixing the frame-cutting task of a single video stream on a frame-cutting machine is likely to cause a single machine to need to bear a large number of long live broadcasts, resulting in problems such as excessive load.

[0059] In an embodiment of the present application, the scheduling cluster can monitor the online live broadcast rooms in the online list to determine whether there is a target live broadcast room that meets the frame capture time condition in the online live broadcast room. If so, it means that the target live broadcast room needs to be frame captured. At this time, a frame capture task corresponding to the target live broadcast room can be generated, so that the frame capture of the live video stream of the online live broadcast room can be uniformly maintained. The frame capture time condition is used to define the frame capture time interval. If a preset time interval has passed since the last frame capture of the target live broadcast room, it can be considered that the target live broadcast room meets the frame capture time condition. In addition, if the target live broadcast room has not been frame captured, if a preset time interval has passed since the target live broadcast room started broadcasting, it can be considered that the target live broadcast room meets the frame capture time condition.

[0060] In other words, the scheduling cluster is equivalent to a timer. After scheduling the online list, the target live broadcast room that needs to be captured can be determined according to the online list. By continuously monitoring the capture time of each live broadcast room in the online list, each online live broadcast room can be uniformly maintained to generate the corresponding capture task for each online live broadcast room. When there are multiple target live broadcast rooms, multiple capture tasks corresponding to multiple target live broadcast rooms can be generated. The capture tasks are independent of each other and can be executed separately, so they can be executed in parallel, which is conducive to improving the efficiency of task scheduling and frame capture efficiency.

[0061] S103, performing a frame capture task on the live video stream of the target live broadcast room through a frame capture module including a plurality of frame capture units to capture a frame capture image corresponding to the target live broadcast room.

[0062] In an embodiment of the present application, a frame capture task can be executed on the live video stream of the target live broadcast room through a frame capture module. Specifically, the scheduling cluster can send a frame capture task to the frame capture module so that the frame capture module executes the frame capture task. The frame capture module may include multiple frame capture units, and the multiple frame capture units constitute a distributed frame capture system, wherein any of the multiple frame capture units is used to execute the frame capture task, that is, for the live video stream of the target live broadcast room, any of the multiple frame capture units can execute its corresponding frame capture task, and the frame capture tasks corresponding to each online live broadcast room are allocated to each frame capture unit, thereby improving load balancing and avoiding the problem of excessive single-machine load caused by uneven distribution of long live broadcasts. In this way, even when there are many online live broadcast rooms resulting in many frame capture tasks, the frame capture tasks can be executed in parallel by multiple frame capture units in the frame capture module, thereby avoiding the task accumulation caused by a single frame capture unit during the execution of the frame capture task, and the task frame capture efficiency can be improved when maintaining multiple live broadcasts.

[0063] In the process of executing the frame capture task, the target unit in the frame capture module including multiple frame capture units can be determined first, and the live video stream of the target live broadcast room can be obtained by pulling the stream from the live content distribution network through the target unit, and then the frame capture task is executed on the live video stream of the target live broadcast room. Figure 4 As shown. The target unit can be any unit among the multiple frame capture units, that is, the frame capture units can be deployed arbitrarily to improve load balancing. Of course, the frame capture units can also be deployed according to certain rules, so that the frame capture tasks assigned to each frame capture unit are more balanced, avoiding the problem of excessive load on a single machine and improving the disaster recovery performance of the system.

[0064] In addition, since there is a time interval for the generation of the frame capture task of the target live broadcast room, when the time interval is short, even if some of the frame capture tasks are not executed, it has little impact on the use scenario of the frame capture image. Therefore, after the target unit obtains the frame capture task, it can terminate the historical frame capture task being executed to execute the new frame capture task. In fact, when the new frame capture task arrives, if the historical frame capture task has not been completed, it means that the historical frame capture task has failed to execute or it takes a long time. Then, terminating the historical frame capture task is conducive to the effective operation of the frame capture unit. Specifically, the target unit can terminate the historical frame capture task being executed, pull the live video stream of the target live broadcast room from the live content distribution network, and execute the frame capture task on the live video stream of the target live broadcast room. Of course, after the execution of this frame capture task is completed, the execution of the historical frame capture task can be restarted, or the historical frame capture task can be abandoned, that is, the execution of the historical frame capture task is not started. Whether the historical frame capture task is abandoned or not can be determined according to the actual situation.

[0065] refer to Figure 7 FIG. 1 is a schematic diagram of a framed image provided in an embodiment of the present application. The framed image is referred to as Figure 7 A~7L, the corresponding frame images of the same live broadcast room are sorted according to the frame capture time, which can reflect the content of the live video stream.

[0066] After obtaining the captured frame image, the captured frame image can be sent to the storage system through the captured frame module so that the storage system can store the captured frame image. Figure 4 As shown, the frame capture module 30 sends the frame capture image to the storage system 160, and the storage system 160 is used to store the frame capture image. The storage system can be a distributed storage system, and the frame capture image can be backed up and stored to improve the storage reliability and stability of the frame capture image and prevent data loss caused by single-machine storage. The storage system can be a cloud storage system that provides cloud storage services, such as a cos storage system. The cos storage system is an object cloud storage service that provides a highly reliable, low-cost, and massively stored cloud storage service with good disaster recovery performance. It is widely used in the Internet, mobile Internet, Internet of Things and other fields, and is a reliable, efficient and secure cloud storage service.

[0067] The storage system can be accessed through a network link to obtain the corresponding frame images in it. The access methods can include HTTP, HTTPS, CNAME, etc., which can be easily integrated with various applications to make the access to the storage system more convenient and secure. Specifically, the query module can obtain the frame images corresponding to the image query link from the query system according to the image query link, so that the frame images can be obtained more conveniently for processing. Figure 4 As shown, the query module 170 can obtain the screenshot image from the storage system 160. The query module 170 can, for example, be an audit module, a live broadcast recommendation module, an internal operation module, etc., wherein the audit module is used to audit the live broadcast room corresponding to the screenshot image based on the screenshot image, the live broadcast recommendation module is used to generate live broadcast recommendation information based on the screenshot image, and the internal operation module is used to generate operation information based on the screenshot image, etc.

[0068] In the embodiment of the present application, the scheduling cluster, online list and frame capture module are separated. The frame capture system composed of the scheduling cluster, online list and frame capture module realizes multi-machine backup and second-level disaster recovery frame capture, which can solve the reliability problem encountered in live video stream frame capture. By scheduling the online list and frame capture module through the scheduling cluster, the system frame capture performance is improved and the system cost is reduced.

[0069] Based on a video stream frame capture method provided in an embodiment of the present application, an embodiment of the present application also provides a video stream frame capture device, referring to Figure 8 , which is a structural block diagram of a video stream frame capture device provided in an embodiment of the present application, the video stream frame capture device 1300 includes:

[0070] The online list acquisition unit 1301 is used to acquire an online list including live broadcast information of an online live broadcast room;

[0071] The frame capture task generating unit 1302 is used to generate a frame capture task corresponding to the target live broadcast room when it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast room;

[0072] The frame capture task execution unit 1303 is used to execute the frame capture task on the live video stream of the target live broadcast room through a frame capture module including multiple frame capture units to capture the frame capture image corresponding to the target live broadcast room. Any one of the multiple frame capture units is used to execute the frame capture task.

[0073] Optionally, the device further comprises:

[0074] An online live broadcast room determination unit, configured to determine an online live broadcast room according to the start broadcast information and the end broadcast information of the live broadcast room before obtaining the online list including the live broadcast information of the online live broadcast room;

[0075] The online list generating unit is used to generate an online list according to the live broadcast information of the online live broadcast room.

[0076] Optionally, the online list is stored in multiple storage nodes, and the online list acquisition unit 1301 is specifically used to:

[0077] An online list stored in a target node among the multiple storage nodes and including live broadcast information of an online live broadcast room is obtained.

[0078] Optionally, the online list includes a key-value pair consisting of a live broadcast identifier of an online live broadcast room and other live broadcast information other than the live broadcast identifier.

[0079] Optionally, the device further comprises:

[0080] The information acquisition unit is used to obtain the start-up information and the end-of-broadcast information of the live broadcast room from the live broadcast server through the live broadcast access layer, and the live broadcast access layer is used to support the live broadcast room to access the network.

[0081] Optionally, the device further comprises:

[0082] A heartbeat packet acquisition unit, used to acquire a heartbeat packet corresponding to the live broadcast room to be detected through the live broadcast access layer, wherein the heartbeat packet includes at least one of an online identifier and interactive information corresponding to the live broadcast room to be detected;

[0083] An updating unit is used to add the live broadcast room to be detected to the online list if the live broadcast room to be detected does not exist in the online list.

[0084] Optionally, the device further comprises:

[0085] The storage unit is used to send the frame-cut image to the storage system through the frame-cut module, so that the storage system backs up and stores the frame-cut image.

[0086] Optionally, the device further comprises:

[0087] The query unit is used to obtain the frame-cut image corresponding to the image query link from the storage system through the query module according to the image query link.

[0088] Optionally, the frame capture task execution unit 1303 includes:

[0089] A target unit determining unit, used to determine a target unit in a frame capturing module including a plurality of frame capturing units;

[0090] The task execution subunit is used to obtain the live video stream of the target live broadcast room by pulling the stream from the live content distribution network through the target unit, and execute the frame capture task on the live video stream of the target live broadcast room.

[0091] Optionally, the task execution subunit is specifically used for:

[0092] The target unit terminates the historical frame capture task being executed, pulls the live video stream of the target live broadcast room from the live content distribution network, and executes the frame capture task on the live video stream of the target live broadcast room.

[0093] Optionally, the device further comprises:

[0094] The sending unit is used to send the online list to the terminal device so that the terminal device displays the online list.

[0095] It can be seen from the above technical solution that the online list can include the live broadcast information of the online live broadcast room. In this way, after obtaining the online list, it can be determined whether there is a target live broadcast room that meets the frame capture time condition in the online live broadcast room according to the online list. If it exists, it means that the target live broadcast room needs to be frame captured. At this time, a frame capture task corresponding to the target live broadcast room can be generated, so that the frame capture of the live video stream of the online live broadcast room can be uniformly maintained. Then, the frame capture task is executed on the live video stream of the target live broadcast room through the frame capture module. The frame capture module includes multiple frame capture units, and the frame capture unit is used to execute the frame capture task. In this way, the frame capture task can be executed by any frame capture unit in the frame capture unit. Even if there are many online live broadcast rooms resulting in many frame capture tasks, the frame capture task is executed in parallel by multiple frame capture units in the frame capture module, avoiding the task accumulation caused by a single frame capture unit during the execution of the frame capture task, and improving the task frame capture efficiency when maintaining multiple live broadcast rooms.

[0096] The embodiment of the present application further provides a computer device, which is the computer device described above and may include a server, which may be connected to a terminal device, and the video stream frame capture device described above may be configured in the computer device. The computer device is described below in conjunction with the accompanying drawings.

[0097] See also Fig. 9 As shown, the embodiment of the present application provides a terminal device, taking the terminal device as a mobile phone as an example:

[0098] Fig. 9 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig. 9 The mobile phone includes: a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490. Those skilled in the art can understand that Fig. 9 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0099] Combine the following Fig. 9 A detailed introduction to the various components of the mobile phone:

[0100] The RF circuit 1410 may be used for receiving and sending signals during information transmission or communication or during a call. In particular, after receiving downlink information from a base station, the information is sent to the processor 1480 for processing. In addition, the uplink data is sent to the base station.

[0101] The memory 1420 can be used to store software programs and modules. The processor 1480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420. The memory 1420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0102] The input unit 1430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the mobile phone. Specifically, the input unit 1430 may include a touch panel 1431 and other input devices 1432 .

[0103] The display unit 1440 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1440 may include a display panel 1441.

[0104] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.

[0105] The audio circuit 1460 , the speaker 1461 , and the microphone 1462 can provide an audio interface between the user and the mobile phone.

[0106] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1470. It provides users with wireless broadband Internet access.

[0107] Processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in memory 1420 and calling data stored in memory 1420.

[0108] The mobile phone also includes a power source 1490 (such as a battery) for supplying power to various components.

[0109] The present application also provides a server. Fig.10 As shown, Fig.10 The structural diagram of the server 1500 provided in the embodiment of the present application, the server 1500 may have relatively large differences due to different configurations or performances, and may include one or more processors 1522, such as central processing units (CPU), memory 1532, and one or more storage media 1530 (e.g., one or more mass storage devices) storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be short-term storage or permanent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the processor 1522 can be configured to communicate with the storage medium 1530 and execute a series of instruction operations in the storage medium 1530 on the server 1500.

[0110] The server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server 2003 or Windows Server 2008. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0111] The steps performed by the server in the above embodiment can be based on Fig.10 The server structure shown.

[0112] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method provided in the above embodiment.

[0113] The embodiments of the present application also provide a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the method provided in the above embodiments.

[0114] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to the computer program, and the aforementioned computer program can be stored in a computer-readable storage medium. When the computer program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned computer-readable storage medium can be at least one of the following media: read-only memory (English: Read-only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store computer programs.

[0115] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0116] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Moreover, based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video stream frame capture method, characterized in that: The method comprises: Get an online list including live broadcast information of online live broadcast rooms; When it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast rooms, a frame capture task corresponding to the target live broadcast room is generated; The frame capture task is performed on the live video stream of the target live broadcast room by a frame capture module including multiple frame capture units to capture the frame capture image corresponding to the target live broadcast room, and any one of the multiple frame capture units is used to perform the frame capture task.

2. The method according to claim 1, characterized in that Before obtaining the online list including the live broadcast information of the online live broadcast room, the method further includes: Determine the online live broadcast room according to the start and end broadcast information of the live broadcast room; Generate an online list based on the live broadcast information of the online live broadcast room.

3. The method according to claim 2, characterized in that The online list is stored in a plurality of storage nodes, and the obtaining of the online list including the live broadcast information of the online live broadcast room comprises: An online list stored in a target node among the multiple storage nodes and including live broadcast information of an online live broadcast room is obtained.

4. The method according to claim 2, characterized in that: The online list includes a key-value pair consisting of a live broadcast identifier of an online live broadcast room and other live broadcast information other than the live broadcast identifier.

5. The method according to claim 2, characterized in that: The method further comprises: The live broadcast access layer obtains the start and end broadcast information of the live broadcast room from the live broadcast server, and the live broadcast access layer is used to support the live broadcast room to access the network.

6. The method according to claim 5, characterized in that The method further comprises: Acquire a heartbeat packet corresponding to the live broadcast room to be detected through the live broadcast access layer, wherein the heartbeat packet includes at least one of an online identifier and interactive information corresponding to the live broadcast room to be detected; If the live broadcast room to be detected does not exist in the online list, the live broadcast room to be detected is added to the online list.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The frame capture module sends the frame capture image to a storage system so that the storage system backs up and stores the frame capture image.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The query module obtains the frame-shot image corresponding to the image query link from the storage system according to the image query link.

9. The method according to any one of claims 1 to 6, characterized in that: The frame capture module including a plurality of frame capture units performs the frame capture task on the live video stream of the target live broadcast room, including: Determining a target unit in a frame capture module including a plurality of frame capture units; The target unit pulls the live video stream of the target live broadcast room from the live content distribution network, and performs the frame capture task on the live video stream of the target live broadcast room.

10. The method according to claim 9, characterized in that The method of obtaining the live video stream of the target live broadcast room by pulling the stream from the live content distribution network through the target unit, and performing the frame capture task on the live video stream of the target live broadcast room, includes: The target unit terminates the historical frame capture task being executed, pulls the live video stream of the target live broadcast room from the live content distribution network, and executes the frame capture task on the live video stream of the target live broadcast room.

11. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The online list is sent to a terminal device so that the terminal device displays the online list.

12. A video stream frame capture device, characterized in that: The device comprises: An online list acquisition unit, used to acquire an online list including live broadcast information of an online live broadcast room; A frame capture task generating unit, configured to generate a frame capture task corresponding to the target live broadcast room when it is determined according to the online list that there is a target live broadcast room that meets the frame capture time condition in the online live broadcast rooms; A frame capture task execution unit is used to execute the frame capture task on the live video stream of the target live broadcast room through a frame capture module including multiple frame capture units to capture the frame capture image corresponding to the target live broadcast room, and any one of the multiple frame capture units is used to execute the frame capture task.

13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the video stream frame capture method described in any one of claims 1-11 according to the instructions in the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the video stream frame capture method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that When the method is run on a computer device, the computer device executes the video stream frame capture method described in any one of claims 1 to 11.

Images