Video processing method, apparatus, device, and storage medium

Abstract

The application provides a video processing method, device and equipment and a storage medium, relates to the technical field of computers, can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving, and the method comprises the following steps: searching for a target fragment task queue from a video fragment queue set in response to a video processing request carrying algorithm stage identification of a video processing algorithm sent by an algorithm server; sending current to-be-scheduled fragment task information in the target fragment task queue to the algorithm server, so that the algorithm server executes stage processing corresponding to the algorithm stage identification on a target video fragment corresponding to the current to-be-scheduled fragment task information based on the video processing algorithm, and obtains a fragment intermediate result; and sending scheduling completion information corresponding to a to-be-processed video to the algorithm server, so that the algorithm server performs merging processing on the fragment intermediate results of a plurality of video fragments corresponding to the to-be-processed video, and obtains an algorithm processing result. The application can improve the video processing efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video processing method, apparatus, device, and storage medium. Background Technology

[0002] In video processing scenarios, it is often necessary to process the original video using one or more algorithms. Existing solutions input the entire original video into each algorithm to obtain its processing result, which is then applied to subsequent tasks. This video processing method uses the original video as the processing granularity, resulting in a high time-to-process ratio and high quota requirements for individual algorithm nodes. If an algorithm node experiences an anomaly, such as node migration, container reconstruction, or node failure, the corresponding algorithm task will fail, requiring the entire original video task to be reprocessed, wasting resources and resulting in low availability. Summary of the Invention

[0003] In view of the above-mentioned existing problems, this application provides a video processing method, apparatus, device and storage medium.

[0004] On the one hand, this application provides a video processing method applied to a video processing terminal, the method comprising:

[0005] In response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the system searches for a target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set. The video segment queue set includes a task queue subset corresponding to at least one video to be processed. The task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed.

[0006] The information of the currently scheduled segment tasks in the target segment task queue is sent to the algorithm server, so that the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segment task information based on the video processing algorithm, and obtains the segment intermediate result.

[0007] The scheduling completion information corresponding to the video to be processed is sent to the algorithm server so that the algorithm server can merge the intermediate results of the multiple video segments corresponding to the video to be processed to obtain the algorithm processing result. The scheduling completion information indicates that there are no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0008] On the other hand, this application provides a video processing method applied to an algorithm server, the method comprising:

[0009] A video processing request carrying an algorithm stage identifier of the video processing algorithm is sent to the video processing terminal, so that the video processing terminal searches for a target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set, and sends the segment task information currently to be scheduled in the target segment task queue to the algorithm server. The video segment queue set includes a task queue subset corresponding to at least one video to be processed, and the task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store the segment task information of multiple video segments of the video to be processed.

[0010] Based on the video processing algorithm, the target video segment corresponding to the current scheduled segmentation task information is processed according to the stage identifier of the algorithm to obtain the intermediate segmentation result.

[0011] In response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, the intermediate results of multiple video segments of the video to be processed are merged to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0012] On the other hand, this application provides a video processing apparatus for use in a video processing terminal, the apparatus comprising:

[0013] Task queue lookup module: In response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the module looks up a target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set. The video segment queue set includes a task queue subset corresponding to at least one video to be processed. The task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed.

[0014] First information sending module: used to send the information of the currently scheduled segment task in the target segment task queue to the algorithm server, so that the algorithm server can perform the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segment task information based on the video processing algorithm, and obtain the segment intermediate result;

[0015] The second information sending module is used to send the scheduling completion information corresponding to the video to be processed to the algorithm server, so that the algorithm server can merge the intermediate results of the multiple video segments corresponding to the video to be processed to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0016] On the other hand, this application provides a video processing apparatus for use on an algorithm server, the apparatus comprising:

[0017] Request sending module: Used to send a video processing request carrying an algorithm stage identifier of the video processing algorithm to the video processing terminal, so that the video processing terminal can find at least one target segment task queue of the video to be processed that matches the algorithm stage identifier from the video segment queue set, and send the segment task information currently to be scheduled in the target segment task queue to the algorithm server. The video segment queue set includes at least one task queue subset corresponding to the video to be processed. The task queue subset includes segment task queues corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed.

[0018] Algorithm processing module: Based on the video processing algorithm, it performs stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segmentation task information to obtain the intermediate segmentation result;

[0019] The first merging processing module is used to merge the intermediate results of multiple video segments of the video to be processed in response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, so as to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0020] On the other hand, a computer device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the video processing method as described above.

[0021] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the video processing method as described above.

[0022] On the other hand, a server is provided, the server including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the video processing method as described above.

[0023] On the other hand, a terminal is provided, the terminal including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the video processing method as described above.

[0024] On the other hand, a computer program product or computer program is provided, which includes computer instructions that, when executed by a processor, implement the video processing method described above.

[0025] The video processing method, apparatus, device, storage medium, server, terminal, computer program, and computer program product provided in this application have the following technical effects:

[0026] This application responds to video processing requests sent by an algorithm server, which carry algorithm stage identifiers for the video processing algorithm. The video processing terminal searches a video segment queue set for at least one target segment task queue of a video to be processed that matches the algorithm stage identifier. The video segment queue set includes a task queue subset corresponding to at least one video to be processed, and this task queue subset includes segment task queues corresponding to at least one algorithm stage of the video processing algorithm. These segment task queues store segment task information for multiple video segments of the video to be processed. Then, the information of the currently scheduled segment tasks in the target segment task queue is sent to the algorithm server. The algorithm server then performs stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segment task information, based on the video processing algorithm, to obtain intermediate segment results. The video processing terminal then searches the task queue subset of the video to be processed for each algorithm stage corresponding to its assigned segment task. If there are no pending video segment tasks in the task queue, the scheduling completion information for the video to be processed is sent to the algorithm server. This allows the algorithm server to merge the intermediate results of multiple video segments corresponding to the video to be processed, thus obtaining the algorithm processing result. In this way, a subset of the task queue is created at the original video granularity. This subset includes the segment task queues created based on each algorithm stage of the video processing algorithm, and stores the task information of the video segments in the queue. This divides the algorithm processing task of the original video into the algorithm stage task granularity of each video segment, significantly reducing the time processing ratio and the configuration requirements of the algorithm nodes in the video processing process. Even if an algorithm node fails, causing the currently executed task to fail, it is only necessary to re-execute the stage processing task of the corresponding video segment. This ensures high availability, avoids resource waste, and enables concurrent atomic monitoring of the video to be processed at the video processing end. Attached Figure Description

[0027] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0029] Figure 2 This is a flowchart illustrating a video processing method provided in an embodiment of this application;

[0030] Figure 3 This is a schematic diagram illustrating the segmentation principle of a video to be processed, provided in an embodiment of this application.

[0031] Figure 4This is a flowchart of a video segmentation process provided in an embodiment of this application;

[0032] Figure 5 This is a storage data structure diagram of a sharded task queue provided in an embodiment of this application;

[0033] Figure 6 This is a flowchart of a segmented task information push process provided in an embodiment of this application;

[0034] Figure 7 This is a flowchart of a fragment task information retrieval method provided in an embodiment of this application;

[0035] Figure 8 This is a flowchart of a fragmented task information processing method provided in an embodiment of this application;

[0036] Figure 9 This is an architecture diagram of an application environment provided in an embodiment of this application;

[0037] Figure 10 This is an architecture diagram of another application environment provided in the embodiments of this application;

[0038] Figure 11 This is a schematic diagram of the frame of a video processing device provided in an embodiment of this application;

[0039] Figure 12 This is a schematic diagram of the frame of another video processing apparatus provided in an embodiment of this application;

[0040] Figure 13 This is a hardware structure block diagram of an electronic device for a video processing method provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0042] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0043] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0044] Time processing ratio: This refers to the ratio between the time it takes for an algorithm to process a video and the length of the video. If a video has a length of 'a' and the time it takes for an algorithm to process the video is 'b', then the time processing ratio of the algorithm is 'b':'a'.

[0045] ETCD is a distributed, highly available, and consistent key-value store database.

[0046] TTL (Time To Live): This value limits the time an IP packet can exist in a computer network.

[0047] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0048] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0049] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application, such as... Figure 1 As shown, the application environment may include at least a video processing terminal 01 and an algorithm processing terminal 02. In practical applications, the video processing terminal 01 and the algorithm processing terminal 02 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.

[0050] In this embodiment, the video processing terminal 01 and the algorithm processing terminal 02 can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0051] Specifically, cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology can be applied to various fields, such as medical cloud, cloud IoT, cloud security, cloud education, cloud conferencing, AI cloud services, cloud applications, cloud calling, and cloud social networking. Based on the cloud computing business model, cloud technology distributes computing tasks across a resource pool composed of numerous computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go. As a provider of basic cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called IaaS (Infrastructure as a Service)) platform is established, deploying various types of virtual resources within the pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0052] Specifically, the video processing terminal 01 and algorithm processing terminal 02 mentioned above may include physical devices, which may specifically include network communication submodules, processors and memory, etc., and may also include software running on the physical devices, which may specifically include applications, etc.

[0053] In this embodiment, the video processing terminal 01 can deploy multiple task scheduling nodes to receive video processing requests sent by the algorithm server. The video processing request carries the algorithm stage identifier of the video processing algorithm, and searches for the target segment task queue based on the algorithm stage identifier. It then sends the information of the currently scheduled segment task in the target segment task queue to the algorithm server 02. Furthermore, it is used to send scheduling completion information to the algorithm server 02 when there is no scheduled segment task information in the segment task queue corresponding to each algorithm stage in the task queue subset of the video to be processed. The algorithm server 02 sends a video processing request to the video processing terminal 01 to trigger the search for the target segment task queue and the sending of the information of the currently scheduled segment task. It then performs stage processing on the target video segment corresponding to the currently scheduled segment task information to obtain the intermediate segment result corresponding to the algorithm stage. In response to the scheduling completion information, it merges the intermediate segment results of multiple video segments of the video to be processed to obtain the algorithm processing result.

[0054] Furthermore, it is understandable that Figure 1 The example shown is merely an application environment for one video processing method. This application environment may include more or fewer nodes, and this application does not impose any limitations on it.

[0055] The application environment involved in this application embodiment, or the video processing terminal 01 and algorithm processing terminal 02 in the application environment, can be a distributed system formed by connecting a client and multiple nodes (any form of computing device accessing the network, such as a server or user terminal) through network communication. The distributed system can be a blockchain system, which can provide the aforementioned video processing services and data storage services.

[0056] The following describes a video processing method based on the aforementioned application environment. This application's embodiments can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating a video processing method provided in an embodiment of this application. This specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as... Figure 2 As shown, the method may include the following steps S201-S207.

[0057] S201: In response to a video processing request sent by the algorithm server, which carries an algorithm stage identifier of the video processing algorithm, the video processing end searches for the target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set.

[0058] In this embodiment of the application, the video segment queue set includes at least one task queue subset corresponding to the video to be processed. The task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store the segment task information of multiple video segments of the video to be processed. The segment task queue is stored in association with the algorithm stage identifier. The algorithm stage identifier is a unique identifier that represents a certain algorithm stage in a video processing algorithm.

[0059] Specifically, the video processing request can be a heartbeat request. Multiple algorithm service nodes can be deployed in the algorithm server. Each algorithm service node can deploy a corresponding video processing algorithm. Each algorithm service node periodically sends a video processing request to the video processing terminal to obtain the segmentation task information of the algorithm stage corresponding to the video processing algorithm in that algorithm service node.

[0060] Specifically, the video processing terminal can deploy one or more task scheduling nodes, and can also deploy a distributed database, which can be built based on ETCD. The distributed database is used to store multiple video segment queue sets. When multiple task scheduling nodes are deployed, each task scheduling node can maintain a corresponding video segment queue set, that is, maintain at least one subset of task queues corresponding to the video to be processed. Specifically, the video processing process of each video processing algorithm can include at least one algorithm stage, and the processing operations of different algorithm stages are mutually exclusive. For example, the face detection algorithm for a video stream includes three algorithm stages: face detection, pre-indexing, and indexing. In a subset of task queues for a video to be processed, a segment task queue is constructed for each of the above three algorithm stages to store the segment task information of each of the multiple video segments in the corresponding video to be processed. The segment task information can include the segment identifier and segment position information of the video segment. This segment task information can be stored in key-value format, with the segment identifier as the key and the segment position information as the value.

[0061] Specifically, in a video processing task, each video processing task (Task) for a video to be processed may include algorithmic processing tasks of one or more video processing algorithms. That is, at least one video processing algorithm is required to process the video to obtain at least one algorithmic processing result. Each video to be processed corresponds to a subset of task queues, and each subset of task queues includes a segmented task queue corresponding to at least one algorithmic stage of the video processing algorithm required for the video to be processed.

[0062] Specifically, during the task scheduling process of the video processing terminal, the task scheduling concurrency is determined based on the number of task scheduling nodes on the video processing terminal and the number of pending videos that can be processed concurrently in each task scheduling node; specifically, the task scheduling concurrency can be the product of the number of task scheduling nodes n and the number of pending videos that can be processed concurrently m.

[0063] Understandably, the concurrency of task scheduling on the video processing end is limited by machine configuration. Therefore, the number of task queue subsets (i.e., m) maintained under each task scheduling node and the number of segmented task queues within each task queue subset are also limited by machine configuration. In some embodiments, after the initial creation of each task queue subset, segmented task queues corresponding to each algorithm stage of a video processing algorithm required for the video to be processed are dynamically created. After all tasks corresponding to any segmented task queue are completed, the segmented task queue is destroyed, and a segmented task queue corresponding to the algorithm stage of another video processing algorithm is created. In other embodiments, a queue concurrency threshold is set, and the number of segmented task queues in the task queue subset must be less than or equal to the concurrency threshold. After all tasks corresponding to any segmented task queue are completed, the segmented task queue is destroyed, and a new segmented task queue is created. The queue creation order can be sequentially created at the granularity of the video processing algorithm.

[0064] In one embodiment, when initially creating a subset of task queues, a segmented task queue corresponding to each algorithm stage of the video processing algorithm is created. For example, the video processing task (Task) for the video to be processed includes the processing tasks of video processing algorithm 1 and video processing algorithm 2. Both video processing algorithm 1 and video processing algorithm 2 include three algorithm stages. Segmented task queues for each of the three algorithm stages of video processing algorithm 1 can be created first. When the tasks in one segmented task queue are completed, the queue is destroyed, and a segmented task queue corresponding to one algorithm stage of video processing algorithm 2 is created.

[0065] In practical applications, after receiving a video processing request from the algorithm server, the video processing terminal, based on the algorithm stage identifier carried by the request, selects the segmented task queues that match the algorithm stage identifier from the various task queue subsets in the video segmented queue set, and identifies these as the target segmented task queues. It is understandable that the video segmented queue set includes multiple task queue subsets of videos to be processed, and if more than one video to be processed may need to call the same video processing algorithm, the same algorithm stage identifier can select segmented task queues from multiple task queue subsets that match the algorithm stage identifier, resulting in multiple target segmented task queues.

[0066] In some embodiments, the video processing terminal includes multiple task scheduling nodes, each task scheduling node corresponding to a set of video segment queues. Specifically, it may correspond to a set of video segment queues stored locally by the task scheduling node. Each task scheduling node only maintains its corresponding set of video segment queues. After the target task scheduling node locks the video processing request, it searches for a segment task queue with an algorithm stage identifier in its corresponding set of video segment queues. If a match is found, it is identified as the target segment task queue. If no match is found, the video processing request is released so that other task scheduling nodes among the multiple task scheduling nodes can lock the video processing request.

[0067] In other embodiments, each task scheduling node maintains a global set of video segment queues for the video processing terminal; accordingly, S201 may include steps S2011-S2012.

[0068] S2011: The video processing terminal controls the target task scheduling node among multiple task scheduling nodes, and searches for the segment task queue that matches the algorithm stage identifier from each task queue subset of the video segment queue set corresponding to the target task scheduling node.

[0069] S2012: If not found, the video processing terminal controls the target task scheduling node to search for a matching segment task queue from the video segment queues corresponding to other task scheduling nodes among the multiple task scheduling nodes.

[0070] Specifically, the video processing end may include a global task table, which can be stored in a distributed database. This global task table includes information about the video segment queue set corresponding to each of the multiple task scheduling nodes. After the target task scheduling node locks onto a video processing request, it first searches its own video segment queue set for a segment task queue that matches the algorithm stage identifier. If a match is found, it is designated as the target segment task queue. If no match is found, the global task table is queried to search the video segment queue sets corresponding to other task scheduling nodes, and the found segment task queue is designated as the target segment task queue. Understandably, if no matching segment task queue is found in the global task table, then the video processing end does not have a stage processing task matching the algorithm stage identifier.

[0071] In practical applications, the video processing end is also used to segment the video to be processed and store the obtained video segments as task information. Accordingly, before S201, the method also includes S301-S305.

[0072] S301: The video processing end performs segmentation processing on the video to be processed, and obtains multiple video segments and segmentation task information for multiple video segments.

[0073] Specifically, the video processing end can segment the video to be processed based on the video slicing service. It can set the preset segment length and the context length of the context information. Each video segment includes context information and a target video segment of the preset segment length. The context information is used for video retrieval processing. Please refer to [reference needed]. Figure 3 , Figure 3 The diagram shows the initial slicing principle of a 0-100s video to be processed. The preset slice length is 20s, and the context length is 2 seconds. That is, the 100s video needs to be divided into 5 target video segments with a preset slice length of 20s, and 2s of context information needs to be retained before and after the target video segment to obtain 5 video slices.

[0074] Specifically, the input information for the video slicing service can include the start time, end time, and video location information of the video to be processed, such as videoStart (milliseconds) / videoEnd (milliseconds) / video URL. After slicing, the video slicing service outputs the segmentation information of multiple video segments of the video to be processed. The segmentation information includes the frame information of the first I-frame and the last P-frame of each video segment.

[0075] In some embodiments, the video segmentation process may include: determining the initial segment start time (segStart) and initial segment end time (segEnd) of multiple video segments in the video to be processed based on the start time, end time, and preset segment length t1. SegStart and segEnd can be in the millisecond range, such as... Figure 3 In the process, the segStart of the five video slices are 0s, 20s, 40s, 60s, and 80s, and the segEnd are 20s, 40s, 60s, 80s, and 100s, respectively. Based on the context length t2, segStart, and segEnd, the initial keyframes of the multiple video slices are determined, and the initial keyframes of the five video slices are 0s, 18s, 38s, 58s, and 78s, respectively. An I-frame is found before each initial keyframe to obtain the target I-frame of each video slice. Based on the preset slice length and context length, multiple video slices are segmented from the video to be processed, starting from each target I-frame. The last frame of each video slice is the first P-frame after segEnd+t2.

[0076] In one embodiment, please refer to Figure 4 , Figure 4 The diagram shows a flowchart of video segmentation processing, which may include the following steps S11-S22.

[0077] S11: Obtain the original video;

[0078] S12: Decompose the original video to obtain the video stream and audio stream to be processed;

[0079] S13: Initialize the video stream to obtain the segStart and segEnd of each of the multiple video slices of the video stream to be processed;

[0080] S14: Determine the initial keyframe ClipRealBeginSec for each video slice in the video to be processed; where the range of the initial keyframe is max(0, segStart-t2), for example, t2 can be 4*(1 / fps), fps is the frame rate of the video to be processed; it can be understood that when ClipRealBeginSec=0, it indicates that the initial keyframe is the first frame of the video to be processed.

[0081] S15: Write a fake frame packet before the initial keyframe ClipRealBeginSec of each video slice, i.e. write a zero frame.

[0082] S16: Determine if the frame time of the initial keyframe ClipRealBeginSec is greater than t2; if it is greater, execute S17; if it is less, execute S20. At this time, the position of ClipRealBeginSec is taken as pts.

[0083] S17: Use the initial keyframe ClipRealBeginSec as the keyframe lookup point seekToSec;

[0084] S18: Search for the initial I-frame in the forward lookup of seekToSec, and define the position of the initial I-frame as pts;

[0085] S19: Determine whether the pts of the initial I-frame is less than seekToSec+2t2 and less than ClipRealBeginSec, and whether the initial I-frame is determined to be an I-frame; if yes, proceed to step S20; if no, proceed to step S21.

[0086] S20: Define seekToSec = current seekToSec - t2, then execute S18;

[0087] Specifically, if an initial I-frame exists in the forward length t2 of seekToSec, its position is defined as pts; if it does not exist, the initial I-frame is searched for in the forward length 2t2 of seekToSec, and so on, until the initial I-frame is found and its position is defined as pts.

[0088] S21: Starting from pts, the initial video slice is segmented from the video to be processed. The last frame of the video slice is the first target P frame after segEnd+t2.

[0089] S22: Add an ending frame after the target P frame of the initial video slice to obtain a video slice.

[0090] Specifically, after obtaining multiple video segments of the video to be processed, they are stored in the cloud storage. The segmentation task information includes the segment identifier and segment location information of the video segment. The segment location information can be the URL of the video segment in the cloud storage.

[0091] S303: The video processing end determines the subset of the target task queue corresponding to the video to be processed from the video segment queue set.

[0092] S305: The video processing end adds the segmentation task information of multiple video segments to the segmentation task queues corresponding to each algorithm stage in the target task queue subset.

[0093] Specifically, each video to be processed is associated with a subset of task queues, and the subset of task queues associated with it is determined as the target task queue subset. The segmented task information is then added to each segmented task queue included in the target task queue subset.

[0094] Understandably, during the initial creation phase of the video processing task for the video to be processed, there may not be a subset of the target task queue, and the method may also include S307-S309.

[0095] S307: If there is no subset of the target task queue in the video segment queue set, the video processing end determines the target video processing algorithm corresponding to the video to be processed and at least one algorithm stage corresponding to the target video processing algorithm.

[0096] S309: The video processing end creates a segmented task queue corresponding to each algorithm stage in at least one algorithm stage, and obtains a subset of the target task queue.

[0097] Specifically, if no subset of the task queue associated with the video to be processed is found, a segmented task queue is created for each algorithm stage corresponding to the video processing algorithm to generate a subset of the target task queue, and then the above step S305 is executed.

[0098] In one embodiment, the data structure of the task queue is as follows: Figure 5As shown, it also includes the number of items. The segment task information in the queue is arranged in order based on the number of items and the key. Specifically, this order can be the temporal order of the video segments in the video to be processed, and the segment task information of the video segments with the earlier temporal order can be preferentially scheduled. The queue task information (Items) includes task information for multiple video segments of a video to be processed, where multiple video segments can represent all video segments of the video to be processed. Status indicates the queue status, including the scheduling status of each task segment in Items, which includes ready (pending processing), doing (processing in progress), and done (completed). Infos represents the queue pointer position, including head, min_ready, tail, and consequent_last_done. Head points to the task with the largest number of items in items, min_ready points to the task with the smallest number of items in status / ready, tail points to the task with the smallest number of items in items, and consequent_last_done points to the position of the completed task in status / done, indicating the position of the completed task with the largest number of items in status / done within Items. Figure 5 The task in this context refers to the fragmented task information.

[0099] In one embodiment, the task scheduling node implements TTL expiration control of the coroutine lock based on a lease mechanism; based on Figure 5 The data structure of the fragmented task queue, as described in S303 to S309 above, involves the task scheduling node pushing fragmented task information to the fragmented task queue as follows: Figure 6 As shown, it includes:

[0100] S21: After the task scheduling node obtains the task information of multiple video segments, it requests a lease; the lease represents the TTL of the coroutine lock.

[0101] S22: Create a coroutine session connection based on the lease ID; the two parties in this coroutine session connection are the task scheduling node and the distributed database;

[0102] S23: Create a coroutine lock based on the coroutine session connection;

[0103] S24: Atomic operations for pushing sharded task information: including: S241. Check if the sharded task queue exists. If it exists, execute S242. If it does not exist, execute S243. S242. Get the head of the sharded task queue (head = Infos -> head). S243. Initialize the queue to create an empty sharded task queue. S244. Push the task to items (Pushtask => items). S245. Record the addition time of the sharded task information (record time => status / ready[head]). S246. Update the header information of items (head + 1 = Infos -> head).

[0104] S25: Unlock the coroutine lock.

[0105] S203: The video processing terminal sends the information of the currently scheduled segment tasks in the target segment task queue to the algorithm server.

[0106] In this embodiment, after the video processing terminal determines the target segment task queue, it checks whether there are any segment task information to be scheduled in the target segment task queue. If so, the segment task information with the highest scheduling order is determined as the current segment task information to be scheduled and sent to the algorithm server. It is understood that if the video processing terminal determines multiple target segment task queues, it can send the current segment task information to be scheduled for each of the multiple target segment task queues to the algorithm server. Specifically, the information of each currently scheduled segment task can be fed back to the algorithm service node that sent the video processing request. Specifically, after sending the currently scheduled segment task information, the video processing terminal updates the target segment task queue.

[0107] In one embodiment, based on Figure 5 The data structure of the fragmented task queue in the code allows the task scheduling node to retrieve the information of the currently scheduled fragmented task from the fragmented task queue as follows: Figure 7 As shown, it includes:

[0108] S31: Request a lease; this lease represents the TTL of the coroutine lock;

[0109] S32: Create a coroutine session connection based on the lease ID; the two parties in this coroutine session connection are the task scheduling node and the distributed database;

[0110] S33: Create a coroutine lock based on the coroutine session connection;

[0111] S34: Atomic operations for retrieving fragmented task information: including: S341. Determine if the fragmented task queue exists. If it exists, execute S342; otherwise, execute S343. S342. Determine if there is any fragmented task information to be scheduled. If it exists, execute S344, where if the head of items >= min_ready, it indicates that there is fragmented task information to be scheduled; otherwise, it indicates that there is no such task, and execute S35. S344. Determine the currently scheduled fragmented task information based on the pointer position information of min_ready in infos (min_ready = infos->min_ready). S345. Retrieve the currently scheduled fragmented task information from items (items[min_ready] => get task). S346. Update the ready information of the scheduled status in status (status / ready[min_ready] => get task). S347. Record the time when the currently scheduled task is marked as being processed (record time => status / doing[min_ready]); S348. Update the pointer position information of min_ready (min_ready+1 => infos->min_ready).

[0112] S35: Unlock the coroutine lock.

[0113] S205: The algorithm server, based on the video processing algorithm, performs the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segmentation task information to obtain the intermediate segmentation result.

[0114] In this embodiment, the algorithm server obtains the target video slice corresponding to the currently scheduled segment task information from the cloud storage based on the received information of the currently scheduled segment task, and performs the stage processing corresponding to the algorithm stage identifier on it. The obtained intermediate segment result is the result of the target video slice after the stage processing of the algorithm stage corresponding to the algorithm stage identifier. For example, if the stage processing operation corresponding to the algorithm stage identifier is the face detection stage in the face detection algorithm, then the intermediate segment result is the result obtained after the face detection stage processing of the target video slice.

[0115] In practical applications, when algorithm processing tasks are divided into stage processing tasks corresponding to each algorithm stage, the algorithm service node on the algorithm server can create multiple algorithm processing services based on the resource usage type corresponding to each algorithm stage. Each algorithm processing service corresponds to a resource usage type, which represents the resources required by the algorithm stage. For example, if CPU or GPU resources are required, a CPU algorithm processing service can be created for the CPU resource usage type, and a GPU algorithm processing service can be created for the GPU resource usage type. The CPU algorithm processing service sends video processing requests corresponding to the algorithm stage with CPU resource usage type, receives feedback segmentation task information, and then calls CPU resources to perform stage processing on the video segments corresponding to the segmentation task information. Similarly, the GPU algorithm processing service sends video processing requests corresponding to the algorithm stage with GPU resource usage type, receives feedback segmentation task information, and calls GPU resources to perform stage processing on the video segments corresponding to the segmentation task information. In this way, the algorithm can decouple and separate the logic of GPU and CPU processing. The GPU algorithm processing service only needs to run the logic that requires GPU computation. After one video segment is processed, the next video segment is processed immediately, significantly improving the resource utilization of different resources.

[0116] After obtaining the intermediate results of the segmentation, the algorithm server also sends a task success feedback to the video processing end. The task success feedback indicates that the processing result of the current segmentation task information stage processing task is successful. After receiving the task success feedback, the video processing end marks the scheduling status of the current segmentation task information as completed, that is, adds it to the completed task queue, and completes the update operation of the completed task queue.

[0117] In one embodiment, based on Figure 5 In the data structure of the fragmented task queue, the task scheduling node responds to the task success feedback, and the operation process of processing the retrieved fragmented task information can be as follows: Figure 8 As shown, it includes:

[0118] S41: Request a lease; this lease represents the TTL of the coroutine lock;

[0119] S42: Create a coroutine session connection based on the lease ID; the two parties in this coroutine session connection are the task scheduling node and the distributed database;

[0120] S43: Create a coroutine lock based on the coroutine session connection;

[0121] S44: Perform atomic operations to process the fragmented task information, including: S441. Update the processing status information of the fragmented task information maintained in status / doing in status, and remove the processing status of the currently scheduled fragmented task information that was previously retrieved (status / doing[index] => get status), where index is equal to min_ready when the fragmented task information was retrieved; S442. Record the time when the currently scheduled fragmented task information is marked as completed (record time => status / done[index]); S443. Update the pointer position information of consequent_last_done (index => infos->consecutive_last_done).

[0122] S45: Unlock the coroutine lock.

[0123] It is understandable that during the process of obtaining target video segments and performing phased processing on the target video segments, abnormal events may occur, such as node migration, container reconstruction or node failure. This may result in the inability to obtain target video segments or to obtain intermediate segmentation results. Accordingly, the method may also include the following steps S401-S403.

[0124] S401: The algorithm server sends the task failure feedback corresponding to the currently scheduled fragment task information.

[0125] S403: In response to task failure feedback, the video processing end marks the scheduling status of the currently scheduled segment task information as pending processing.

[0126] Specifically, the task failure feedback indicates that the processing result of the stage processing task of the currently scheduled segment task information is a task failure; the task failure feedback is generated when the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment and does not obtain the intermediate result of the segment.

[0127] Specifically, initially, the scheduling status of the currently scheduled task segments is "pending processing." After sending the information to the algorithm server, the task scheduling node in the video processing terminal marks the scheduling status of the task segment as "completed" if it receives a successful task feedback; otherwise, it re-marks it as "pending processing" to instruct the video processing terminal to resend subsequent video processing requests carrying the algorithm stage identifier to the algorithm server for reprocessing the corresponding stage. In this way, by using the task scheduling node to handle task retry logic and error recording and reporting, when reproducing errors and rerunning tasks, only intermediate results need to be stored. The video slice where the error occurred is used to request the corresponding logical unit of the algorithm module where the error occurred, reducing the cost of task retry.

[0128] Specifically, after remarking the currently scheduled task fragments as pending, a new item for that task fragment can be added to the queue task information items. For example, initially, the number of items for the currently scheduled task fragments is 5, and the maximum number of items in the queue task information items is 10. After receiving task failure feedback for the currently scheduled task fragments, a new item item11 is added to facilitate subsequent rescheduling. The number of reschedulings for each task fragment is less than or equal to the scheduling count threshold, such as 3 times. If the task success feedback is not received after exceeding the scheduling count threshold, it is marked as an abnormal task, and an alarm message is generated.

[0129] S207: The video processing terminal sends the scheduling completion information corresponding to the video to be processed to the algorithm server.

[0130] In this embodiment, the scheduling completion information indicates that there are no unscheduled segment tasks in the segment task queues of each algorithm stage corresponding to the video processing algorithm. This means that for the current video to be processed, the task processing status of all segment tasks corresponding to all algorithm stages of the current video processing algorithm is successful, and the video segment corresponding to each segment task has obtained intermediate segmentation results. Accordingly, before S207, the method further includes the following steps S501-S503.

[0131] S501: For each video to be processed in at least one video to be processed, the video processing end monitors the task reserve status corresponding to the video processing algorithm.

[0132] Specifically, the task reserve status represents the number of unscheduled task information in each segment task queue of each algorithm stage corresponding to the video processing algorithm. The video processing end can monitor the number of unscheduled task information in the segment task queue of each algorithm stage based on a monitoring thread. In some embodiments, the number of unscheduled task information in the segment task queue of each algorithm stage can be monitored based on a watch mechanism, and the monitoring thread can dynamically obtain changes in the number of unscheduled task information through the watch interface.

[0133] In one embodiment, the number of sharded task information to be scheduled in each sharded task queue can be determined based on the queue pointer position. The head position of the sharded task queue is the sharded task information with the largest number of items in the queue. If the position of the sharded task information with the scheduling status of completed is the same as the head position, it is determined that there is no sharded task information to be scheduled in the sharded task queue. If the positions are different, the difference between the head position and the position of the completed sharded task information is the number of sharded task information to be scheduled in the queue.

[0134] S503: If the task reserve status is that there are no segmented task information to be scheduled, the video processing terminal generates scheduling completion information.

[0135] Specifically, if any video to be processed does not have any scheduled segment task information in the segment task queue corresponding to all algorithm stages of the video processing algorithm, the scheduling completion information of the video to be processed under the video processing algorithm is generated and sent to the algorithm server to trigger S209.

[0136] S209: In response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, the algorithm server merges the intermediate results of multiple video segments of the video to be processed to obtain the algorithm processing result.

[0137] In this embodiment, the algorithm server, based on scheduling completion information, determines the stage of processing each algorithm phase of the video processing algorithm, performing stage processing on all video segments of the video to be processed, and then merging the intermediate results of the segments. Understandably, this merging process can select all or part of the intermediate segment results for merging based on actual needs to obtain the algorithm processing result.

[0138] In summary, a subset of task queues is created at the original video granularity. This subset includes segmented task queues created based on each algorithm stage of the video processing algorithm, and the task information of the video segments is stored in the queues. This divides the algorithm processing task of the original video into algorithm stage task granularity for each video segment, significantly reducing the time processing ratio and the configuration requirements of algorithm nodes in the video processing process. Even if an algorithm node malfunctions, causing the currently executed task to fail, it is only necessary to re-execute the stage processing task of the corresponding video segment. This ensures high availability, avoids resource waste, and enables concurrent atomic monitoring of the video to be processed at the video processing end.

[0139] In some embodiments, for each video to be processed, after the processing of all video segments in one algorithm stage is completed, the result merging processing of that algorithm stage can be triggered. Accordingly, the method also includes the following steps S505-S507.

[0140] S505: If the task reserve status is that there are no unscheduled segment task information in the segment task queue of any algorithm stage in the video processing algorithm, the video processing terminal sends the stage task completion information to the algorithm server.

[0141] S507: In response to the completion information of the stage task, the algorithm server merges the intermediate results of the segments of the video to be processed that correspond to the algorithm stage to obtain the stage processing result of the video to be processed.

[0142] Specifically, the stage task completion information indicates that a certain algorithm stage of the video processing algorithm has completed the stage processing task of all video segments in the video to be processed, and obtained intermediate results for each segment. That is, the task status of the segment task information for this algorithm stage is "task successful." After receiving this stage task completion information, the algorithm server determines the algorithm stage corresponding to the stage task completion information and merges the intermediate results generated from the stage processing of all video segments in the video to be processed to obtain the stage processing result. Understandably, this merging process can select all or part of the intermediate results for merging based on actual needs. In some cases, S209 can be the algorithm server responding to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, merging the stage processing results of multiple video segments of the video to be processed to obtain the algorithm processing result.

[0143] In one embodiment, each task scheduling node maintains a local video segment queue set. Video processing tasks received by a task scheduling node are processed only on that node; other nodes do not receive tasks from that node. Figure 5 The data structure of the task queue in the image, and the architecture of the application environment for the video processing method in this application are as follows: Figure 9As shown, a task scheduling node can simultaneously establish streaming connections with multiple algorithm service nodes, meaning it can handle multiple video processing tasks concurrently. A stage represents an algorithm phase; the number of stages corresponds to the number of segment task queues maintained per video processing task. Queue information can be obtained by accessing the aforementioned queue key. Assuming the video processing algorithm has three stages, each task requires three watches. Each watch corresponds one-to-one with a stage and is attached to the queue key, monitoring changes in the number of segment tasks awaiting scheduling in the segment task queues of that stage. Figure 9 In the process, task scheduling nodes 1, 2, and 3 respectively push the segmented task information for Task1, 2, and 3. After receiving the video processing request sent by the algorithm service node, the task scheduling node determines the target segmented task information from its local video processing set and sends it to the corresponding algorithm service node, and receives task feedback (task success feedback or task failure feedback). Figure 9 In the process, the algorithm stage identifier of the video processing request currently received by task scheduling node 1 is stage1. Task scheduling node 1 pushes video segment 1 and segment 2 of Task1 and receives task success feedback (Done) for video segment 1 and segment 2 of Task1. The algorithm stage identifier of the video processing request currently received by task scheduling node 2 is stage1. Task scheduling node 2 pushes video segment 1 of Task2 and receives task success feedback (Done) for video segment 1 of Task2.

[0144] In another embodiment, each task scheduling node maintains a global set of video segment queues. Tasks received by the current task scheduling node can be scheduled by other task scheduling nodes. Each task scheduling node prioritizes processing its own tasks, followed by tasks from other task scheduling nodes. Figure 5 The data structure of the task queue in the image, and the architecture of the application environment for the video processing method in this application are as follows: Figure 10 As shown, this architecture is similar to the distributed architecture in the diagram. The difference is that after receiving a message from the message queue, the task scheduling node first records it in the etcd global task key-value table (Etcd global task) to update the etcd global task. When retrieving segment task information from the video segment queue, it will prioritize processing its own local tasks. If there are no local tasks, it will process the global tasks in other nodes.

[0145] Furthermore, the architecture described above is highly reusable and can be applied to general AI algorithm capabilities. As long as the AI ​​algorithm supports video segmentation processing, it can be applied to this architecture for accelerated processing, especially time-consuming algorithms such as face recognition, human segmentation, and OCR.

[0146] Based on the video processing method of this application, the availability, fault tolerance, resource utilization, and performance of video processing will be significantly improved. The GPU quota of a single video processing algorithm node can be reduced to below 30%, the GPU memory usage can be reduced to below 7%, and the number of CPU cores can be reduced to 4 cores. For example, in a face recognition algorithm without video slicing, processing a complete 45-minute video requires one T4 GPU card, 122GB of memory, and a 32-core CPU for each algorithm service node. With this solution, a single algorithm service node only requires 0.3 T4 GPU cards, 8GB of memory, and a 4-core CPU. Furthermore, in face recognition and face segmentation algorithms, the time processing ratio of existing technologies is 1:1. With this solution, the time processing ratio of high-definition and ultra-high-definition videos can reach 1 / 6 to 1 / 7 of existing technologies, the time processing ratio of standard-definition videos can reach 1 / 10, and the time processing ratio of standard-definition videos for person segmentation is reduced to 1 / 10.

[0147] This application also provides a video processing method applied to a video processing terminal, which may include the following steps.

[0148] S601: In response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, search for at least one target segment task queue of the video to be processed that matches the algorithm stage identifier from the video segment queue set. The video segment queue set includes at least one task queue subset corresponding to the video to be processed. The task queue subset includes segment task queues corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed.

[0149] S603: Send the information of the currently scheduled segmentation tasks in the target segmentation task queue to the algorithm server, so that the algorithm server can perform the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segmentation task information based on the video processing algorithm, and obtain the intermediate segmentation result.

[0150] S605: Send the scheduling completion information corresponding to the video to be processed to the algorithm server so that the algorithm server can merge the intermediate results of multiple video segments corresponding to the video to be processed to obtain the algorithm processing result. The scheduling completion information indicates that there are no segment tasks to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0151] In some embodiments, prior to S605, the method further includes:

[0152] S701: For each video to be processed in at least one video to be processed, monitor the task reserve status corresponding to the video processing algorithm. The task reserve status represents the number of segmented task information to be scheduled in each segmented task queue of each algorithm stage corresponding to the video processing algorithm.

[0153] S703: If the task reserve status is that there are no fragmented task information to be scheduled, generate scheduling completion information.

[0154] In some embodiments, the method further includes:

[0155] S705: If the task reserve status is that there are no unscheduled segment task information in the segment task queue of any algorithm stage in the video processing algorithm, send the stage task completion information to the algorithm server so that the algorithm server can merge the segment intermediate results of the video to be processed corresponding to the algorithm stage to obtain the stage processing result of the video to be processed.

[0156] In some embodiments, prior to S601, the method further includes:

[0157] S801: Perform segmentation processing on the video to be processed to obtain multiple video segments and segmentation task information for multiple video segments;

[0158] S803: Determine the subset of the target task queue corresponding to the video to be processed from the set of video segment queues;

[0159] S805: Add the segmentation task information of multiple video segments to the segmentation task queues corresponding to each algorithm stage in the target task queue subset.

[0160] In some embodiments, the method further includes:

[0161] S807: If there is no subset of the target task queue in the video segmentation queue set, determine the target video processing algorithm corresponding to the video to be processed and at least one algorithm stage corresponding to the target video processing algorithm.

[0162] S809: Create a task queue corresponding to each algorithm stage in at least one algorithm stage to obtain a subset of the target task queue.

[0163] In some embodiments, the method further includes S901: in response to the task failure feedback corresponding to the currently scheduled segment task information sent by the algorithm server, the scheduling status of the currently scheduled segment task information is marked as pending processing status; the task failure feedback is generated when the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment and does not obtain the intermediate segment result.

[0164] In some embodiments, the video processing terminal includes multiple task scheduling nodes, each task scheduling node corresponding to a video segment queue set; S601 includes:

[0165] S6011: Controls the target task scheduling node among multiple task scheduling nodes, and searches for the segment task queue that matches the algorithm stage identifier from each task queue subset of the video segment queue set corresponding to the target task scheduling node.

[0166] S6012: If not found, control the target task scheduling node to search for a matching segment task queue from the video segment queue set corresponding to other task scheduling nodes among the multiple task scheduling nodes.

[0167] This application also provides a video processing method applied to an algorithm server, which may include the following steps.

[0168] S1001: Send a video processing request carrying the algorithm stage identifier of the video processing algorithm to the video processing terminal, so that the video processing terminal can find the target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set, and send the segment task information currently to be scheduled in the target segment task queue to the algorithm server. The video segment queue set includes at least one task queue subset corresponding to the video to be processed. The task queue subset includes the segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store the segment task information of multiple video segments of the video to be processed.

[0169] S1003: Based on the video processing algorithm, perform the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the current segmentation task information to be scheduled, and obtain the intermediate segmentation result;

[0170] S1005: In response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, the intermediate results of the segmentation of multiple video segments of the video to be processed are merged to obtain the algorithm processing result. The scheduling completion information indicates that there are no segmentation task information to be scheduled in the segmentation task queue of each algorithm stage corresponding to the video processing algorithm.

[0171] In some embodiments, the method further includes S1007: if a stage task completion information is received from the video processing terminal, the intermediate results of the segments of the video to be processed corresponding to the algorithm stage are merged to obtain the stage processing result of the video to be processed.

[0172] The stage task completion information is generated by the video processing end in response to the fact that there are no unscheduled segment task information in the segment task queue of any algorithm stage of the video processing algorithm when the task reserve status is in the segment task queue of each algorithm stage. The task reserve status represents the number of unscheduled segment task information in each segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0173] In some embodiments, the method further includes:

[0174] S1009: If no intermediate segmentation result is obtained after performing the stage processing corresponding to the algorithm stage identifier on the target video segmentation, the task failure feedback corresponding to the generated current segmentation task information to be scheduled will be provided.

[0175] S1011: Send task failure feedback to the video processing end so that the video processing end marks the scheduling status of the currently scheduled segment task information as pending processing.

[0176] This application also provides a video processing apparatus, applied to a video processing terminal, such as... Figure 11 As shown, Figure 11 The diagram shows a structural schematic of a video processing apparatus provided in an embodiment of this application. The apparatus may include the following modules.

[0177] Task queue lookup module 11: In response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the module looks up the target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set. The video segment queue set includes a task queue subset corresponding to at least one video to be processed. The task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store the segment task information of multiple video segments of the video to be processed.

[0178] First information sending module 12: Used to send the information of the currently scheduled segmented task in the target segmented task queue to the algorithm server, so that the algorithm server can perform the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segmented task information based on the video processing algorithm, and obtain the intermediate segmented result.

[0179] The second information sending module 13 is used to send the scheduling completion information corresponding to the video to be processed to the algorithm server, so that the algorithm server can merge the intermediate results of multiple video segments corresponding to the video to be processed to obtain the algorithm processing result. The scheduling completion information indicates that there are no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0180] In some embodiments, the apparatus further includes:

[0181] Monitoring module: Before sending the scheduling completion information corresponding to the video to be processed to the algorithm server, for each video to be processed in at least one video to be processed, the monitoring module monitors the task reserve status corresponding to the video processing algorithm. The task reserve status represents the number of segment task information to be scheduled in each segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0182] Scheduling completion information generation module: Used to generate scheduling completion information if the task reserve status is that there are no unscheduled task fragments.

[0183] In some embodiments, the apparatus further includes a stage information sending module: if the task reserve status is that there is no segment task information to be scheduled in the segment task queue of any algorithm stage of the video processing algorithm, the module sends stage task completion information to the algorithm server so that the algorithm server merges the segment intermediate results of the video to be processed corresponding to the algorithm stage to obtain the stage processing result of the video to be processed.

[0184] In some embodiments, the apparatus further includes:

[0185] Video Segmentation Module: Before responding to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the module segments the video to be processed into multiple video segments and segmentation task information of multiple video segments by searching the target segmentation task queue of at least one video to be processed that matches the algorithm stage identifier from the video segmentation queue set.

[0186] Queue Subset Determination Module: Used to determine the target task queue subset corresponding to the video to be processed from the video segment queue set;

[0187] Task Addition Module: Used to add the task information of multiple video segments to the corresponding segment task queues of each algorithm stage in the target task queue subset.

[0188] In some embodiments, the apparatus further includes:

[0189] Algorithm determination module: If there is no subset of the target task queue in the video segment queue set, determine the target video processing algorithm corresponding to the video to be processed and at least one algorithm stage corresponding to the target video processing algorithm.

[0190] Queue creation module: Used to create a segmented task queue corresponding to each algorithm stage in at least one algorithm stage, thereby obtaining a subset of the target task queue.

[0191] In some embodiments, the apparatus further includes a status marking module: used to mark the scheduling status of the currently scheduled segment task information as pending processing status in response to the task failure feedback corresponding to the currently scheduled segment task information sent by the algorithm server; the task failure feedback is generated when the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment and does not obtain the intermediate segment result.

[0192] In some embodiments, the video processing terminal includes multiple task scheduling nodes, each task scheduling node corresponding to a video segment queue set; the task queue lookup module 11 includes:

[0193] First search unit: Used to control the target task scheduling node among multiple task scheduling nodes, and search for the segment task queue that matches the algorithm stage identifier from each task queue subset of the video segment queue set corresponding to the target task scheduling node.

[0194] The second search unit is used to control the target task scheduling node to search for a matching segment task queue from the video segment queues corresponding to other task scheduling nodes among multiple task scheduling nodes if no matching task queue is found.

[0195] This application also provides a video processing device 12, applied to an algorithm server, such as... Figure 12 As shown, Figure 12 The diagram shows a structural schematic of a video processing apparatus provided in an embodiment of this application. The apparatus may include the following modules.

[0196] Request sending module 21: Used to send a video processing request carrying the algorithm stage identifier of the video processing algorithm to the video processing end, so that the video processing end can find the target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set, and send the segment task information currently to be scheduled in the target segment task queue to the algorithm server. The video segment queue set includes at least one task queue subset corresponding to the video to be processed. The task queue subset includes the segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store the segment task information of multiple video segments of the video to be processed.

[0197] Algorithm processing module 22: Based on the video processing algorithm, it performs the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segmentation task information to obtain the intermediate segmentation result;

[0198] The first merging processing module 23 is used to merge the intermediate results of multiple video segments of the video to be processed in response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, so as to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0199] In some embodiments, the apparatus further includes a second merging processing module: if it receives a stage task completion information sent by the video processing terminal, it merges the intermediate results of the segments of the video to be processed corresponding to the algorithm stage to obtain the stage processing result of the video to be processed.

[0200] The stage task completion information is generated by the video processing end in response to the fact that there are no unscheduled segment task information in the segment task queue of any algorithm stage of the video processing algorithm when the task reserve status is in the segment task queue of each algorithm stage. The task reserve status represents the number of unscheduled segment task information in each segment task queue of each algorithm stage corresponding to the video processing algorithm.

[0201] In some embodiments, the apparatus further includes:

[0202] Failure feedback module: Used to generate task failure feedback corresponding to the current scheduled segmentation task information if no intermediate segmentation result is obtained after performing the stage processing corresponding to the algorithm stage identifier on the target video segmentation;

[0203] Scheduling status marking module: Used to send task failure feedback to the video processing end, so that the video processing end marks the scheduling status of the currently scheduled segment task information as pending processing.

[0204] It should be noted that the above-described device embodiments and method embodiments are based on the same implementation methods.

[0205] This application provides a video processing device, which can be a terminal or a server, including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the video processing method provided in the above method embodiments.

[0206] Memory can be used to store software programs and modules. The processor executes various functional applications and video processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created based on device usage, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0207] The methods and embodiments provided in this application can be executed in electronic devices such as mobile terminals, computer terminals, servers, or similar computing devices. Figure 13 This is a hardware structure block diagram of an electronic device for a video processing method provided in an embodiment of this application. For example... Figure 13 As shown, the electronic device 900 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module including a series of instruction operations on the electronic device. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute a series of instruction operations in the storage media 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server. TM Mac OS X TM Unix TM Linux™, FreeBSD™, etc.

[0208] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 900. In one example, the input / output interface 940 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0209] Those skilled in the art will understand that Figure 13 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 900 may also include... Figure 13 The more or fewer components shown, or having the same Figure 13 The different configurations shown.

[0210] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a video processing method in the method embodiments. The at least one instruction or the at least one program is loaded and executed by the processor to implement the video processing method provided in the above method embodiments.

[0211] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0212] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0213] As can be seen from the embodiments of the video processing method, apparatus, device, server, terminal, storage medium, and program product provided in this application, this application responds to a video processing request sent by an algorithm server, carrying an algorithm stage identifier of the video processing algorithm, through a video processing terminal. It searches a video segment queue set for at least one target segment task queue of a video to be processed that matches the algorithm stage identifier. The video segment queue set includes a task queue subset corresponding to at least one video to be processed, and the task queue subset includes segment task queues corresponding to at least one algorithm stage of the video processing algorithm. The segment task queues are used to store segment task information for multiple video segments of the video to be processed. Then, the segment task information currently to be scheduled in the target segment task queue is sent to the algorithm server, so that the algorithm server, based on the video processing algorithm, performs stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segment task information, obtaining intermediate segment results. The video processing terminal... If there are no pending task information in the task queues corresponding to the algorithm stages of each algorithm stage within the task queue subset of the video to be processed, the scheduling completion information for the video to be processed is sent to the algorithm server. This allows the algorithm server to merge the intermediate results of multiple video segments corresponding to the video to be processed, thus obtaining the algorithm processing result. In this way, a task queue subset is created at the original video granularity. This subset includes the task queues created based on each algorithm stage of the video processing algorithm, and the task information of the video segments is stored in the queues. This divides the algorithm processing task of the original video into the task granularity of each video segment's algorithm stage, significantly reducing the time processing ratio and the configuration requirements of the algorithm nodes in the video processing process. Even if an algorithm node malfunctions, causing the currently executed task to fail, it is only necessary to re-execute the stage processing task of the corresponding video segment. This ensures high availability, avoids resource waste, and enables concurrent atomic monitoring of the video to be processed at the video processing end.

[0214] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0215] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0216] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0217] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A video processing method, applied to a video processing terminal, characterized in that, The method includes: In response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the system searches for a target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set. The video segment queue set includes a task queue subset corresponding to at least one video to be processed. The task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed. The information of the currently scheduled segment tasks in the target segment task queue is sent to the algorithm server, so that the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segment task information based on the video processing algorithm, and obtains the segment intermediate result. The scheduling completion information corresponding to the video to be processed is sent to the algorithm server so that the algorithm server can merge the intermediate results of the multiple video segments corresponding to the video to be processed to obtain the algorithm processing result. The scheduling completion information indicates that there are no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

2. The method according to claim 1, characterized in that, Before sending the scheduling completion information corresponding to the video to be processed to the algorithm server, the method further includes: For each of the at least one videos to be processed, the task reserve status corresponding to the video processing algorithm is monitored. The task reserve status represents the number of segmented task information to be scheduled in each segmented task queue of each algorithm stage corresponding to the video processing algorithm. If the task availability status indicates that there is no information on the sharded task to be scheduled, then the scheduling completion information is generated.

3. The method according to claim 2, characterized in that, The method further includes: If the task reserve status is such that there is no scheduled segment task information in the segment task queue of any algorithm stage in the video processing algorithm, the stage task completion information is sent to the algorithm server so that the algorithm server can merge the segment intermediate results of the video to be processed corresponding to the algorithm stage to obtain the stage processing result of the video to be processed.

4. The method according to claim 1, characterized in that, Before searching the target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set in response to a video processing request carrying an algorithm stage identifier sent by the algorithm server, the method further includes: The video to be processed is segmented to obtain multiple video segments and segmentation task information for the multiple video segments; A subset of the target task queues corresponding to the videos to be processed is determined from the video segment queue set; The segmentation task information of the multiple video segments is added to the segmentation task queue corresponding to each algorithm stage in the target task queue subset.

5. The method according to claim 4, characterized in that, The method further includes: If the target task queue subset does not exist in the video segmentation queue set, determine the target video processing algorithm corresponding to the video to be processed and at least one algorithm stage corresponding to the target video processing algorithm. Create a segmented task queue corresponding to each of the at least one algorithm stage to obtain the target task queue subset.

6. The method according to claim 1, characterized in that, The method further includes: In response to the task failure feedback corresponding to the currently scheduled shard task information sent by the algorithm server, the scheduling status of the currently scheduled shard task information is marked as pending processing status. The task failure feedback is generated when the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment and fails to obtain the intermediate result of the segment.

7. The method according to claim 1, characterized in that, The video processing terminal includes multiple task scheduling nodes, each task scheduling node corresponding to a video segment queue set; the step of searching for the target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set includes: Control the target task scheduling node among the multiple task scheduling nodes, and search for the segment task queue that matches the algorithm stage identifier from each task queue subset of the video segment queue set corresponding to the target task scheduling node. If not found, control the target task scheduling node to search for the matching segment task queue from the video segment queue set corresponding to other task scheduling nodes among the multiple task scheduling nodes.

8. A video processing method, applied to an algorithm server, characterized in that, The method includes: A video processing request carrying an algorithm stage identifier of the video processing algorithm is sent to the video processing terminal, so that the video processing terminal searches for a target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set, and sends the segment task information currently to be scheduled in the target segment task queue to the algorithm server. The video segment queue set includes a task queue subset corresponding to at least one video to be processed, and the task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store the segment task information of multiple video segments of the video to be processed. Based on the video processing algorithm, the target video segment corresponding to the current scheduled segmentation task information is processed according to the stage identifier of the algorithm to obtain the intermediate segmentation result. In response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, the intermediate results of multiple video segments of the video to be processed are merged to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

9. A video processing apparatus, applied to a video processing terminal, characterized in that, The device includes: Task queue lookup module: In response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the module looks up a target segment task queue of at least one video to be processed that matches the algorithm stage identifier from the video segment queue set. The video segment queue set includes a task queue subset corresponding to at least one video to be processed. The task queue subset includes a segment task queue corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed. First information sending module: used to send the information of the currently scheduled segment task in the target segment task queue to the algorithm server, so that the algorithm server can perform the stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segment task information based on the video processing algorithm, and obtain the segment intermediate result; The second information sending module is used to send the scheduling completion information corresponding to the video to be processed to the algorithm server, so that the algorithm server can merge the intermediate results of the multiple video segments corresponding to the video to be processed to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

10. The apparatus according to claim 9, characterized in that, The device further includes: Monitoring module: Before sending the scheduling completion information corresponding to the video to be processed to the algorithm server, for each of the at least one videos to be processed, the monitoring module monitors the task balance status corresponding to the video processing algorithm. The task balance status represents the number of segment task information to be scheduled in each segment task queue of each algorithm stage corresponding to the video processing algorithm. Scheduling completion information generation module: used to generate scheduling completion information if the task reserve status is that there is no information on the sharded task to be scheduled.

11. The apparatus according to claim 10, characterized in that, The device further includes: Stage Information Sending Module: If the task reserve status is such that there is no segment task information to be scheduled in the segment task queue of any algorithm stage in the video processing algorithm, the stage task completion information is sent to the algorithm server so that the algorithm server can merge the intermediate segment results of the video to be processed corresponding to the algorithm stage to obtain the stage processing result of the video to be processed.

12. The apparatus according to claim 9, characterized in that, The device further includes: Video Segmentation Module: Before searching for the target segmentation task queue of at least one video to be processed that matches the algorithm stage identifier in response to a video processing request sent by the algorithm server carrying an algorithm stage identifier of the video processing algorithm, the module segments the video to be processed into multiple video segments and segmentation task information of the multiple video segments. Queue subset determination module: used to determine the target task queue subset corresponding to the video to be processed from the video segment queue set; Task Addition Module: Used to add the segmentation task information of the multiple video segments to the segmentation task queues corresponding to each algorithm stage in the target task queue subset.

13. The apparatus according to claim 12, characterized in that, The device further includes: Algorithm determination module: used to determine the target video processing algorithm corresponding to the video to be processed and at least one algorithm stage corresponding to the target video processing algorithm if the target task queue subset does not exist in the video segment queue set; Queue creation module: used to create a segmented task queue corresponding to each algorithm stage in the at least one algorithm stage, so as to obtain the target task queue subset.

14. The apparatus according to claim 9, characterized in that, The device further includes: Status marking module: In response to the task failure feedback corresponding to the currently scheduled task information sent by the algorithm server, the scheduling status of the currently scheduled task information is marked as pending. The task failure feedback is generated when the algorithm server performs the stage processing corresponding to the algorithm stage identifier on the target video segment and fails to obtain the intermediate result of the segment.

15. The apparatus according to claim 9, characterized in that, The video processing terminal includes multiple task scheduling nodes, and each task scheduling node corresponds to a video segment queue set. The task queue lookup module includes: First search unit: used to control the target task scheduling node among the plurality of task scheduling nodes to search for a segment task queue that matches the algorithm stage identifier from each task queue subset of the video segment queue set corresponding to the target task scheduling node. The second search unit is used to control the target task scheduling node to search for the matching segment task queue from the video segment queue set corresponding to other task scheduling nodes among the multiple task scheduling nodes if no match is found.

16. A video processing apparatus, applied to an algorithm server, characterized in that, The device includes: Request sending module: Used to send a video processing request carrying an algorithm stage identifier of the video processing algorithm to the video processing terminal, so that the video processing terminal can find at least one target segment task queue of the video to be processed that matches the algorithm stage identifier from the video segment queue set, and send the segment task information currently to be scheduled in the target segment task queue to the algorithm server. The video segment queue set includes at least one task queue subset corresponding to the video to be processed. The task queue subset includes segment task queues corresponding to at least one algorithm stage of the video processing algorithm. The segment task queue is used to store segment task information of multiple video segments of the video to be processed. Algorithm processing module: Based on the video processing algorithm, it performs stage processing corresponding to the algorithm stage identifier on the target video segment corresponding to the currently scheduled segmentation task information to obtain the intermediate segmentation result; The first merging processing module is used to merge the intermediate results of multiple video segments of the video to be processed in response to the scheduling completion information corresponding to the video to be processed sent by the video processing terminal, so as to obtain the algorithm processing result. The scheduling completion information indicates that there is no segment task information to be scheduled in the segment task queue of each algorithm stage corresponding to the video processing algorithm.

17. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the video processing method as described in any one of claims 1-8.

18. A computer device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the video processing method as described in any one of claims 1-8.

19. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, implement the video processing method as described in any one of claims 1-8.

Images