Distributed video transcoding method and device, equipment and storage medium

Abstract

The invention discloses a distributed video transcoding method and device, equipment and a storage medium. The method is applied to a transcoding server in a distributed system, the distributed system comprises the transcoding server and a plurality of transcoding clients, and the method comprises the following steps: receiving a video transcoding task sent by a service system; the video transcoding task is decomposed into a plurality of sub-tasks of the video to be transcoded, the sub-tasks are stored in a sub-task table, and different sub-tasks are used for processing different types of metadata of the video to be transcoded; according to the current equipment state of each transcoding client and a scheduling strategy in a preset strategy table, scheduling the sub-tasks which are not scheduled in the sub-task table to the transcoding client meeting the scheduling strategy; and after task execution results of all the subtasks of the to-be-transcoded video are received, generating a transcoding result of the to-be-transcoded video, and sending the transcoding result to the service system. Therefore, distributed transcoding can be realized, and transcoding efficiency and processing capability are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a distributed video transcoding method, device, equipment and storage medium. Background Art

[0002] With the development of the Internet and mobile communication technologies, the dissemination and consumption of video content are increasing. Video transcoding is the process of converting video content into different formats or encoding standards to adapt to different playback devices and network environments. Transcoding technology is crucial to the distribution of video content and user experience.

[0003] The distributed video transcoding solution provided by related technologies usually divides the video into slices and then distributes them to multiple transcoding servers for processing to achieve the effect of distributed transcoding. However, when processing large-scale video transcoding tasks, this solution is prone to bottlenecks in computing resources and is difficult to effectively improve transcoding efficiency. It is also difficult to expand and apply to other types of transcoding scenarios. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a distributed video transcoding method, apparatus, device and storage medium. The technical solution of the present invention is implemented as follows:

[0005] In a first aspect, a distributed video transcoding method is provided, which is applied to a transcoding server in a distributed system, wherein the distributed system includes a transcoding server and multiple transcoding clients, and the method includes:

[0006] Receive video transcoding tasks sent by the business system;

[0007] Decomposing the video transcoding task into a plurality of subtasks of the video to be transcoded, and storing the plurality of subtasks in a subtask table, wherein different subtasks are used to process different types of metadata of the video to be transcoded;

[0008] According to the current device status of each of the transcoding clients and the scheduling policy in the preset policy table, the unscheduled subtasks in the subtask table are scheduled to the transcoding clients that meet the scheduling policy;

[0009] After receiving the task execution results of all subtasks of the video to be transcoded, a transcoding result of the video to be transcoded is generated, and the transcoding result is sent to the business system.

[0010] In some embodiments, the video transcoding task is decomposed into a plurality of subtasks of the video to be transcoded, including:

[0011] Determining a target configuration corresponding to the business system from a preset transcoding template;

[0012] According to transcoding parameters of different types of metadata of the video to be transcoded in the target configuration, the video transcoding task is decomposed into a plurality of subtasks of the video to be transcoded.

[0013] In some embodiments, according to the current device status of each of the transcoding clients and the scheduling policy in the preset policy table, scheduling the unscheduled subtasks in the subtask table to the transcoding clients that meet the scheduling policy includes:

[0014] For any of the transcoding clients, upon receiving a task request from the transcoding client, judging whether the transcoding client satisfies all scheduling policies corresponding to the transcoding client in the preset policy table according to the current device state of the transcoding client, and obtaining a judgment result;

[0015] If the judgment result is yes, determining the unscheduled subtasks in the subtask table;

[0016] The transcoding client is used as a target transcoding client, and the unscheduled subtasks in the subtask table are scheduled to the target transcoding client.

[0017] In some embodiments, the scheduling strategy corresponding to the transcoding client includes at least one of the following:

[0018] A first policy, used to indicate a time period during which the transcoding client is allowed to execute a task;

[0019] A second policy is used to indicate a CPU usage threshold when the transcoding client is allowed to execute a task;

[0020] A third policy is used to indicate a memory usage threshold when the transcoding client is allowed to execute a task;

[0021] A fourth strategy is used to instruct the transcoding client to execute a task of a preset priority;

[0022] A fifth policy includes a state value of a policy switch of the transcoding client, wherein the state value of the policy switch is used to indicate whether to perform task scheduling on the transcoding client.

[0023] In some embodiments, storing the plurality of subtasks in the subtask table includes:

[0024] For each subtask stored in the subtask table, a distributed lock is added to the subtask, wherein the distributed lock is used to prevent multiple transcoding clients from processing the same subtask at the same time;

[0025] The determining of the unscheduled subtasks in the subtask table includes:

[0026] The unscheduled subtasks in the subtask table are determined by distributed locks.

[0027] In some embodiments, the method further comprises:

[0028] In the case where there are unscheduled subtasks in the subtask table, if each of the transcoding clients is in a state of executing tasks, the unscheduled subtasks in the subtask table are scheduled to a designated transcoding engine for processing according to a preset timed task.

[0029] In some embodiments, the method further comprises:

[0030] For each subtask of the video to be transcoded, monitoring the running status of the transcoding client processing the subtask;

[0031] When it is monitored that the operating state of any one of the transcoding clients is abnormal, the subtask of the abnormally operating transcoding client is scheduled to the normally operating transcoding client.

[0032] In a second aspect, a distributed video transcoding device is provided, which is applied to a transcoding server in a distributed system, wherein the distributed system includes a transcoding server and multiple transcoding clients, and the device includes:

[0033] A receiving module, used to receive video transcoding tasks sent by the business system;

[0034] A decomposition module, used for decomposing the video transcoding task into a plurality of subtasks of the video to be transcoded, wherein different subtasks are used for processing different types of metadata of the video to be transcoded;

[0035] A storage module, used for storing the plurality of subtasks in a subtask table;

[0036] A scheduling module, configured to schedule unscheduled subtasks in the subtask table to transcoding clients that meet the scheduling strategy according to the current device status of each transcoding client and the scheduling strategy in the preset strategy table;

[0037] A generating module, configured to generate a transcoding result of the video to be transcoded after receiving the task execution results of all subtasks of the video to be transcoded;

[0038] A sending module is used to send the transcoding result to the business system.

[0039] In some embodiments, the decomposition module is used to:

[0040] Determining a target configuration corresponding to the business system from a preset transcoding template;

[0041] According to transcoding parameters of different types of metadata of the video to be transcoded in the target configuration, the video transcoding task is decomposed into a plurality of subtasks of the video to be transcoded.

[0042] In some embodiments, the scheduling module is used to:

[0043] For any of the transcoding clients, upon receiving a task request from the transcoding client, judging whether the transcoding client satisfies all scheduling policies corresponding to the transcoding client in the preset policy table according to the current device state of the transcoding client, and obtaining a judgment result;

[0044] If the judgment result is yes, determining the unscheduled subtasks in the subtask table;

[0045] The transcoding client is used as a target transcoding client, and the unscheduled subtasks in the subtask table are scheduled to the target transcoding client.

[0046] In some embodiments, the scheduling strategy corresponding to the transcoding client includes at least one of the following:

[0047] A first policy, used to indicate a time period during which the transcoding client is allowed to execute a task;

[0048] A second policy is used to indicate a CPU usage threshold when the transcoding client is allowed to execute a task;

[0049] A third policy is used to indicate a memory usage threshold when the transcoding client is allowed to execute a task;

[0050] A fourth strategy is used to instruct the transcoding client to execute a task of a preset priority;

[0051] A fifth policy includes a state value of a policy switch of the transcoding client, wherein the state value of the policy switch is used to indicate whether to perform task scheduling on the transcoding client.

[0052] In some embodiments, the storage module is used to:

[0053] For each subtask stored in the subtask table, a distributed lock is added to the subtask, wherein the distributed lock is used to prevent multiple transcoding clients from processing the same subtask at the same time;

[0054] The scheduling module is used to:

[0055] The unscheduled subtasks in the subtask table are determined by distributed locks.

[0056] In some embodiments, the scheduling module is further configured to:

[0057] In the case where there are unscheduled subtasks in the subtask table, if each of the transcoding clients is in a state of executing tasks, the unscheduled subtasks in the subtask table are scheduled to a designated transcoding engine for processing according to a preset timed task.

[0058] In some embodiments, the apparatus further comprises: a monitoring module;

[0059] The monitoring module is used to monitor the running status of the transcoding client processing the subtask for each subtask of the video to be transcoded;

[0060] The scheduling module is further configured to schedule the subtask of the abnormally operating transcoding client to the normally operating transcoding client when it is monitored that the operating status of any one of the transcoding clients is abnormal.

[0061] According to a third aspect, an electronic device is provided, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein when the processor runs the computer program, the steps of the distributed video transcoding method provided in any one of the aforementioned embodiments are executed.

[0062] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions; after the computer-executable instructions are executed by a processor, the steps of the distributed video transcoding method provided in any one of the aforementioned embodiments can be implemented.

[0063] In the technical solution provided by the embodiment of the present invention, the embodiment of the present invention provides a distributed video transcoding method, which is different from the solution of segmenting the transcoded video file. Instead, the video transcoding task is decomposed into different types of video metadata processing tasks, which are distributed to different transcoding clients through a distributed framework for parallel processing. In this way, distributed transcoding can be achieved, which is suitable for large-scale video transcoding tasks and scenarios such as tight transcoding server resources. The complexity is low and the system is easy to expand, which effectively improves the transcoding efficiency and processing capacity; in addition, by combining the current device status of each transcoding client and the scheduling strategy in the preset strategy table, the unscheduled subtasks in the subtask table are scheduled, which can achieve the purpose of reasonable allocation and effective utilization of device resources and optimize resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic flow chart of a distributed video transcoding method provided by an embodiment of the present invention.

[0065] Figure 2 A schematic flow chart of a distributed video transcoding method provided by an embodiment of the present invention.

[0066] Figure 3 A schematic diagram of a flow chart of a distributed video transcoding method provided by an embodiment of the present invention.

[0067] Figure 4 A schematic diagram of the architecture of a distributed system provided by an embodiment of the present invention.

[0068] Figure 5 A timing diagram of a distributed video transcoding method provided by an embodiment of the present invention.

[0069] Figure 6 A schematic diagram of the structure of a distributed video transcoding device provided by an embodiment of the present invention.

[0070] Figure 7 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0072] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0073] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0075] Video transcoding refers to converting a compressed and encoded video stream into another video stream to adapt to different network bandwidths, different terminal processing capabilities, and different user needs. Transcoding is essentially a process of decoding first and then encoding, so the streams before and after the conversion may or may not follow the same video coding standard. The current mainstream video coding standards include H.264 / AVC, H.265 / HEVC, VP9, ​​and AV1. These coding standards have their own advantages in compression efficiency and video quality, but the transcoding process usually requires higher computing resources. ffmpeg is a widely used open source multimedia framework that supports conversion, streaming, and playback of multiple video and audio formats. ffmpeg has become one of the main tools in the field of video transcoding due to its powerful functions and flexibility.

[0076] With the popularity of high-resolution videos such as 4K and 8K, and the emergence of emerging applications such as virtual reality (VR) and augmented reality (AR), video transcoding faces higher computing requirements and more complex processing tasks. At the same time, users have higher and higher requirements for video loading speed and playback smoothness.

[0077] The idea of ​​a distributed transcoding solution provided by the related technology is to divide the video into slices, and then distribute them to multiple transcoding servers for processing to achieve the effect of distributed transcoding, that is, cut the video file into multiple file blocks (for example, using tools such as mkvmerg), distribute these file blocks to different transcoding servers for processing, and after all the file blocks are processed, merge the files to finally synthesize the transcoded files. Although this solution can achieve distributed transcoding and improve the utilization of server resources when the transcoding server resources are sufficient, it also has some defects. For example, there is no improvement effect when there are many video transcoding requests and the transcoding server resources are scarce, and the implementation is complex, the system is not easy to expand, and the cutting and merging of files also takes extra time.

[0078] In some embodiments, a flow chart of a distributed video transcoding method is provided. Figure 1 , the method may include the following steps:

[0079] Step S101: receiving a video transcoding task sent by a business system;

[0080] Step S102: decomposing the video transcoding task into multiple subtasks of the video to be transcoded, and storing the multiple subtasks in a subtask table, wherein different subtasks are used to process different types of metadata of the video to be transcoded;

[0081] Step S103: according to the current device status of each transcoding client and the scheduling policy in the preset policy table, scheduling the unscheduled subtasks in the subtask table to the transcoding client that meets the scheduling policy;

[0082] Step S104: after receiving the task execution results of all subtasks of the video to be transcoded, generating a transcoding result of the video to be transcoded, and sending the transcoding result to the business system.

[0083] The distributed video transcoding method provided by the embodiment of the present invention is applied to the transcoding server in the distributed system, and the distributed system includes the transcoding server and multiple transcoding clients. The transcoding server is mainly used to accept transcoding tasks submitted by the third-party business system, and is responsible for the classification, decomposition, scheduling, detection, callback, etc. of the tasks; the transcoding client is a plurality of server nodes deployed in a distributed manner, and is used to pull and execute transcoding tasks in the mixed deployment scenario of the formal production environment and the test environment.

[0084] In some examples, step S101 may include: the transcoding server receives a video transcoding task sent by a business system by calling a specified API interface of the transcoding server. Here, there may be multiple business systems, and different business systems correspond to different business fields. For example, the multiple business systems are an Internet live broadcast system, a video surveillance system, and a video social system.

[0085] In some examples, the video transcoding task is used to instruct to transcode the video to be transcoded in the business system. The task may involve changing the original encoding format, resolution, bit rate and other parameters of the video to be transcoded to adapt to the playback requirements of different terminals and network environments.

[0086] In some examples, step S102 may include: the transcoding server may decompose the video transcoding task of the business system into multiple subtasks of the videos to be transcoded according to the transcoding configuration corresponding to the business system, and store them in the subtask table.

[0087] Here, each subtask in the subtask table has a unique ID and task information, and the ID of the subtask is associated with the ID of the main task (i.e., the video transcoding task). The task information of the subtask may include the subtask status (such as pending, processing, completed, failed, etc.) and task time information (such as creation time, start time, end time).

[0088] In some examples, step S103 may include: the transcoding server matches the current device status of each transcoding client with the scheduling policy in the preset policy table, determines the transcoding client that meets the scheduling policy, and schedules the unscheduled subtasks in the subtask table to the transcoding client that meets the scheduling policy.

[0089] Different business systems may correspond to different transcoding configurations. Here, the transcoding configuration is a set of transcoding parameters and settings predefined for video transcoding. For example, the transcoding configuration specifies at least one of the information such as the definition of the video to be transcoded, the position and size of the frame, the audio bit rate, the audio sampling rate, the audio decoder, the text transcription engine, the text transcription format, and the subtitle transcription format.

[0090] In actual applications, different transcoding configurations can be created for the same business system in the transcoding configuration template according to different requirements and scenarios. For example, the transcoding configuration corresponding to the business system can be determined in the transcoding configuration corresponding to the business system according to the ID and scenario parameters of the business system.

[0091] In some examples, different types of metadata of the video to be transcoded may include: video, voice, frame, text, subtitles, etc. The transcoding server may split the video transcoding task of the business system into different subtasks for processing different types of metadata according to the transcoding configuration corresponding to the business system.

[0092] In some examples, the device status of the transcoding client may include: task execution status and device performance status. The task execution status is used to characterize the status of the transcoding task currently being executed or completed by the transcoding client, for example, it may include the progress and status of the task (such as waiting for transcoding, transcoding, transcoding failure, etc.), and may also include information such as the transcoding output path and transcoding time. The device performance status is used to characterize the usage of the device resources of the transcoding client (such as CPU load, memory load).

[0093] In some examples, the device status of the transcoding client can be reported by the transcoding client to the transcoding server through a heartbeat reporting mechanism, where the heartbeat reporting mechanism is that the transcoding client sends device status information to the transcoding server periodically (for example, every 5 seconds).

[0094] In some examples, the preset policy table may include multiple scheduling policies, and the scheduling policies are used to control the scheduling of unscheduled subtasks in the subtasks. The scheduling policies may include: scheduling policies based on device priority, scheduling policies based on events (for example, allowing scheduling tasks when the device status changes), scheduling based on rules (for example, setting fixed time intervals or time points to allow scheduling tasks), etc.

[0095] Here, different transcoding clients correspond to different scheduling strategies. For example, two different transcoding clients are configured with device priority strategies, one of which is configured with a device priority strategy that allows the execution of tasks with the highest priority, and the other is configured with a device priority strategy that allows the execution of tasks with the lowest priority.

[0096] In some embodiments, the subtask table is stored in a database, and scheduling the unscheduled subtasks in the subtask table to the transcoding clients that meet the scheduling policy according to the current device status of each transcoding client and the scheduling policy in the preset policy table includes:

[0097] The transcoding server takes the transcoding client that meets the scheduling strategy as the target transcoding client according to the current device status of each transcoding client and the scheduling strategy in the preset strategy table, and updates the task information of the subtask assigned to the target transcoding client in the subtask table. The updated task information of the subtask includes the allocation information of the subtask assigned to the target transcoding client; the subtask table is used for the target transcoding client to determine the subtask assigned to the target transcoding client after monitoring the allocation information of the subtask.

[0098] In some embodiments, the method further comprises:

[0099] The transcoding server monitors the execution status changes of each subtask in the subtask table to determine the execution status of each subtask; the execution status of the subtask may include, for example: executing, timed out, execution failed, execution completed, etc.

[0100] In this embodiment, the subtask table can be stored in a database, and the transcoding server and each transcoding client can monitor the subtask table in the database in real time. For example, the transcoding client executes the subtask when it monitors that there is a subtask assigned to the transcoding client in the subtask table. The transcoding client can update the task execution status of the subtask executed by itself to the subtask table, so that the transcoding server can determine the execution status of each subtask by monitoring the execution status changes of each subtask in the subtask table.

[0101] In some examples, the transcoding server can monitor the status changes of the subtask table by polling or event notification. By monitoring the status changes of the subtask table, the transcoding server can understand the execution status of each subtask in real time. If a subtask times out or fails, the transcoding server needs to reallocate the task to another client or mark the entire main task as failed.

[0102] In some examples, each transcoding client can run one or more listening services, which are used to query or listen to the subtask table in the database. The listening service can use polling (regularly querying the database) or event-driven methods to detect subtasks newly added to the subtask table or subtasks after the task information in the subtask table is updated. When the transcoding client detects the subtask assigned to it, it will start executing the task. During the execution process, the transcoding client can update the status of the subtask as needed (such as from "not executed" to "executing"). If the subtask is successfully completed, the transcoding client updates the status of the subtask in the subtask table to "completed", and can store the path of the result or output file in the preset field contained in the task information of the subtask. If the task fails or times out, the transcoding client updates the status of the subtask in the subtask table to "failed" or "timeout" and records the error details.

[0103] In this embodiment, by monitoring the subtask table by the transcoding server and each transcoding client, efficient management and execution of transcoding tasks can be achieved while ensuring the accuracy and consistency of data.

[0104] In some examples, step S104 may include: after the transcoding server sends the subtask to the transcoding client that meets the scheduling strategy, the transcoding server receives the task execution result of the transcoding client, wherein the task execution result is obtained by the transcoding client calling the local transcoding tool (e.g., ffmpeg) to execute the processing of the subtask, for example, executing transcoding, frame capture, voice transcription, text transcription, or subtitle processing, etc. After the transcoding client executes the subtask processing, it calls the task result reporting interface to report the subtask execution result to the transcoding server. The transcoding server monitors and collects the task execution results of all subtasks, and merges the results of all subtasks into a complete transcoding result as the transcoding result of the video to be transcoded, and sends the transcoding result to the business system. The merging may include: synchronizing the audio, subtitle and video streams, etc.

[0105] The embodiment of the present invention provides a distributed video transcoding method. Different from the scheme of segmenting the transcoded video file, the method decomposes the video transcoding task into different types of video metadata processing tasks, which are distributed to different transcoding clients through a distributed framework for parallel processing. In this way, distributed transcoding can be achieved, which is suitable for large-scale video transcoding tasks and scenarios such as tight transcoding server resources. The method has low complexity and is easy to expand the system, which effectively improves the transcoding efficiency and processing capacity. In addition, by combining the current device status of each transcoding client and the scheduling strategy in the preset strategy table, the unscheduled subtasks in the subtask table are scheduled, which can achieve the purpose of reasonable allocation and effective utilization of device resources and optimize resource utilization.

[0106] In some embodiments, Figure 2 As shown, the above step S102 decomposes the video transcoding task into multiple subtasks of the video to be transcoded, which may include the following steps:

[0107] Step S201: determining a target configuration corresponding to a business system from a preset transcoding template;

[0108] Step S202: Decomposing the video transcoding task into a plurality of subtasks of the video to be transcoded according to the transcoding parameters of different types of metadata of the video to be transcoded in the target configuration.

[0109] In some examples, the transcoding template may include transcoding configuration information in XML format.

[0110] In some examples, the transcoding configuration information is bound to the ID of the business system, and the target configuration corresponding to the business system may include multiple transcoding parameters, including but not limited to: the clarity of the video to be transcoded (for example, 480P, 720P, 1080P), the frame position (for example, the position of the captured frame is marked as 10) and the frame size (for example, 576*370), the audio bit rate (for example, 128kbps), the audio sampling rate (for example, 44100Hz), the audio decoder (for example, libmp3lame, i.e., the MP3 codec library), the text transcription format, etc.

[0111] In some examples, in order to meet more transcoding requirements and scenarios, the above-mentioned frame position can be marked as the frame position of a specific event, action or expression, that is, the position of the captured frame is not directly marked as a specific frame number.

[0112] In some examples, the above step S202 may include: decomposing the video transcoding task into multiple subtasks according to the transcoding parameters required for transcoding different types of metadata of the video to be transcoded in the target configuration, and different subtasks perform transcoding processing on different types of metadata of the video to be transcoded.

[0113] In the embodiment of the present invention, the video transcoding task is decomposed into different types of video metadata processing tasks, the system scalability is enhanced, and more video transcoding tasks are supported. On the basis of video transcoding, it can be achieved that the processing capabilities of voice, frame capture, text, subtitles, etc. are expanded at the same time, which can meet the application of large-scale video transcoding tasks and scenarios such as tight transcoding server resources.

[0114] In some embodiments, Figure 3 As shown, according to the current device status of each transcoding client and the scheduling policy in the preset policy table, scheduling the unscheduled subtasks in the subtask table to the transcoding client that meets the scheduling policy may include the following steps:

[0115] Step S301: for any transcoding client, when receiving a task request from the transcoding client, judging whether the transcoding client satisfies all scheduling policies corresponding to the transcoding client in the preset policy table according to the current device state of the transcoding client, and obtaining a judgment result;

[0116] Step S302: if the judgment result is yes, determine the unscheduled subtasks in the subtask table;

[0117] Step S303: taking the transcoding client as the target transcoding client, scheduling the unscheduled subtasks in the subtask table to the target transcoding client.

[0118] In some examples, step S301 may include: the transcoding server receives a task request from a transcoding client, the request includes a device code parameter (e.g., device identifier) ​​of the transcoding client; according to the device code parameter, obtains the current device state of the transcoding client from a device state table, the client device state table stores device state data reported to the transcoding server by each transcoding client through a heartbeat reporting mechanism, such as CPU usage, memory occupancy, device temperature, task execution status, etc. According to the device code parameter of the transcoding client, search for all scheduling policies corresponding to the transcoding client in a preset policy table. The scheduling policy table defines task scheduling policies for different transcoding clients. Traverse all scheduling policies corresponding to the transcoding client, and determine one by one whether the current device state of the transcoding client meets each scheduling policy, for example, compare the actual values ​​of CPU usage, memory occupancy, etc. with the thresholds set in the scheduling policy. If the current device state of the transcoding client meets the requirements of all scheduling policies, a judgment result of meeting the scheduling policy is generated. If any one or more device state data does not meet the requirements of at least one scheduling policy, it is determined that the transcoding client does not meet all scheduling policies, and a judgment result of not meeting the scheduling policy is generated.

[0119] In some examples, step S302 may include: when the judgment result indicates that the current device state of the client satisfies all corresponding scheduling policies, determining the unscheduled subtasks in the subtask table according to the execution state of each subtask in the subtask table.

[0120] In the embodiment of the present invention, by executing the above steps, it is determined whether the transcoding client that initiates the task request satisfies all corresponding scheduling strategies in the preset strategy table. In this way, the use of rich scheduling strategies can bring multi-dimensional control functions to the transcoding process, making transcoding more flexible, efficient and reliable.

[0121] In some embodiments, the scheduling strategy corresponding to the transcoding client includes at least one of the following:

[0122] A first policy is used to indicate a time period during which the transcoding client is allowed to execute tasks;

[0123] The second policy is used to indicate the CPU usage threshold when the transcoding client is allowed to execute the task;

[0124] The third policy is used to indicate a memory usage threshold when the transcoding client is allowed to execute a task;

[0125] A fourth strategy is used to instruct the transcoding client to execute a task of a preset priority;

[0126] A fifth policy includes a state value of a policy switch of the transcoding client, wherein the state value of the policy switch is used to indicate whether to perform task scheduling on the transcoding client.

[0127] In some examples, the first policy corresponding to the transcoding client is recorded as IP_TIME_SLOT, which is used to control the time period for the transcoding client to execute tasks. If the current time does not meet the requirements, no task will be allocated to the transcoding client.

[0128] In some examples, the second policy corresponding to the transcoding client is recorded as CPU_THRESHOLD, which is used to limit the CPU threshold of the device serving as the transcoding client. When the transcoding client device reports a heartbeat, it will be accompanied by the current CPU usage. If it exceeds the threshold, no task will be assigned to the transcoding client.

[0129] In some examples, the third policy corresponding to the transcoding client is recorded as MEM_THRESHOLD, which is used as the device memory threshold limit of the transcoding client. When the transcoding client device reports a heartbeat, it will be accompanied by the current memory usage. If it exceeds the threshold, no task will be assigned to the transcoding client.

[0130] In some examples, the fourth policy corresponding to the transcoding client is denoted as PRIORITY, which is used to set the device priority. If the policy is specified, the transcoding client device will execute the task of the priority. The video transcoding task submitted by the business system can carry the priority of the task.

[0131] In some examples, the fifth policy corresponding to the transcoding client is recorded as SWITCH, which is used as a scheduling switch. For example, if it is set to off, no task is assigned to the transcoding client.

[0132] In some examples, the scheduling strategy corresponding to the transcoding client may also include a sixth strategy, which can be recorded as MACHINE_COMMAND. The strategy contains custom transcoding instructions (ffmpeg transcoding parameters). Generally speaking, there are many combinations of transcoding client devices in the production environment, such as local servers, test servers, cloud hosts, docker containers, etc. The graphics card driver versions of various servers are different, so the transcoding instructions will be different. This strategy specifies the transcoding instructions used by a certain transcoding client device.

[0133] In some embodiments, storing multiple subtasks in the subtask table in step S102 may include:

[0134] Each time a subtask is stored in the subtask table, a distributed lock is added to the subtask, wherein the distributed lock is used to prevent multiple transcoding clients from processing the same subtask at the same time;

[0135] In the above step S302, determining the unscheduled subtasks in the subtask table may include:

[0136] The unscheduled subtasks in the subtask table are determined through distributed locks.

[0137] In some examples, distributed locks can be implemented based on Redis, Zookeeper, or a database.

[0138] In some examples, when a new subtask is added to the subtask table, a distributed lock is created for the subtask. The ID of the subtask is associated with the distributed lock. When the transcoding client requests to pull a subtask, the transcoding server first checks whether the requested subtask has been locked. If the subtask has been locked, the transcoding server returns an error message or asks the client to wait until the lock is released. If the subtask is not locked, the transcoding server locks the subtask and assigns it to the transcoding client that sent the task request. After receiving the subtask, the transcoding client starts executing the task and holds the distributed lock during the task execution. When the transcoding client completes the task, it sends a completion signal to the transcoding server and releases the distributed lock. After receiving the completion signal, the transcoding server updates the status of the subtask and confirms that the lock has been released.

[0139] In some examples, if the transcoding server detects that the distributed lock assigned to the transcoding client (e.g., the first client) has timed out, the lock is released again, and other clients (e.g., the second client) can request and process the subtask. The distributed lock has a timeout mechanism. For example, if the transcoding client fails or crashes during the task execution, causing the distributed lock to time out, the transcoding server will automatically release the distributed lock for the subtask.

[0140] The embodiment of the present invention can effectively avoid the problem of multiple transcoding clients processing the same subtask at the same time by adding a distributed lock to the subtask, ensuring the correct execution of the task and the integrity of the data. At the same time, by monitoring the status and usage of the distributed lock and determining the unscheduled subtasks in the subtask table, the stability and performance of the system can be ensured.

[0141] In some embodiments, the method further comprises:

[0142] In the case that there are unscheduled subtasks in the subtask table, if each transcoding client is in a state of executing tasks, the unscheduled subtasks in the subtask table are scheduled to the designated transcoding engine for processing according to the preset timed tasks.

[0143] In this embodiment, when a scheduled task is triggered, the transcoding server can check whether there are unscheduled subtasks in the subtask table through the task status information. If there are unscheduled subtasks, the status of all transcoding clients is checked through the heartbeat signal sent by the client to confirm whether they are all in the state of executing tasks. If all clients are in the state of executing tasks, the transcoding server selects a designated transcoding engine to process these unscheduled subtasks. Here, the designated transcoding engine can be a spare, high-priority or special processing capability transcoding engine, such as a cloud transcoding engine.

[0144] For example, the transcoding server can send a notification to the specified transcoding engine by sending a message to a message queue, calling an API interface, or sending a network request, schedule the subtask to the specified transcoding engine for processing, and monitor the processing progress and results of the subtask by the specified transcoding engine.

[0145] In this way, by being compatible with local transcoding and external cloud transcoding control, when the local transcoding cluster is busy, external cloud transcoding services can be used for processing at the same time, thereby ensuring timely processing of subtasks.

[0146] In some embodiments, the method further comprises:

[0147] For each subtask of the video to be transcoded, the running status of the transcoding client processing the subtask is monitored; when the running status of any transcoding client is monitored to be abnormal, the subtask of the abnormal transcoding client is scheduled to the normal transcoding client.

[0148] In this embodiment, the transcoding server can detect whether any transcoding client is running abnormally based on the received client status information. Here, the abnormality may include client crash, network disconnection, processing timeout, etc. In the case of detecting that a certain client is running abnormally, the transcoding server determines the subtasks that the client is currently processing, removes these subtasks from the queue of the abnormal client, and reallocates them to other transcoding clients that are running normally. At the same time, when reallocating tasks, the server will update the status of the subtask to ensure the accuracy and consistency of the task status. In this way, the transcoding server can automatically reschedule tasks when an abnormality occurs in the transcoding client, thereby ensuring the continuous processing of subtasks and improving the overall stability of the system.

[0149] Combine the following Figure 4 , Figure 5 The distributed transcoding method provided by the embodiment of the present invention is further described.

[0150] In some embodiments, a distributed transcoding method is provided. The method is applied to a distributed system and can process transcoding tasks in parallel on multiple computing nodes, thereby improving transcoding efficiency, reducing costs, and providing better scalability and flexibility.

[0151] like Figure 4 As shown in the figure, the distributed system includes a transcoding server and a transcoding client. The transcoding server is mainly used to accept transcoding tasks submitted by third-party business systems, and is responsible for task classification, disassembly, scheduling, detection, callback, etc. The transcoding client is a distributed deployment of multiple server nodes, mixed in the formal and test environments, and is mainly used to pull and execute transcoding tasks.

[0152] like Figure 4 , Figure 5 As shown in the figure, the specific interaction process between the transcoding server and the transcoding client in the distributed system is as follows:

[0153] 1. First, preset the transcoding template (for example, preset the transcoding template on the transcoding server). Specify the tag in the template content (for example, <apps>Tag) declares the transcoding configuration of all business systems, which can cover detailed configurations such as text, voice, video, frame capture, subtitles, etc.

[0154] 2. When receiving a business request, the transcoding server obtains the app ID from the request and parses it into the transcoding configuration in the corresponding template. The transcoding configuration specifies the definition of the video to be transcoded, the frame position and size, the audio bit rate, the audio sampling rate, the audio decoder, the text transcription engine, the text transcription format, the subtitle transcription format, etc.

[0155] 3. The transcoding server breaks down the video, voice, frame capture, text, subtitles, etc. into different subtasks according to the parsed template configuration and puts them into the database subtask table.

[0156] 4. Each transcoding client will report heartbeats regularly, which include information such as CPU load, memory load, and whether there are tasks to execute (to mark whether the client is idle).

[0157] 5. The transcoding client calls the transcoding server to fetch subtasks at preset intervals (e.g., 10 seconds), and the scheduling module of the transcoding server assigns subtasks to the transcoding client for execution.

[0158] 6. The scheduling module of the transcoding server allocates tasks according to the overall load.

[0159] If there is no task to execute, the subtask is directly assigned.

[0160] If all clients have tasks to execute, the scheduled task will directly assign the subtasks to the Alibaba Cloud transcoding engine for execution.

[0161] If the CPU load exceeds the threshold (specified in the policy table) or the memory load exceeds the threshold (specified in the policy table), no tasks will be assigned to the client to avoid the avalanche effect.

[0162] 7. After the transcoding client pulls the task at a fixed time interval, it calls the local ffmpeg to perform transcoding, frame capture, voice and other processing. After the processing is completed, it calls the task result reporting interface to report the subtask processing results.

[0163] When the transcoding client starts, it reads a local fixed device file, which stores the device ID, that is, a UUID, which is generated by the first startup of the client and is used for unique identification of the transcoding server.

[0164] The transcoding client will report the device status at fixed intervals, including device ID, graphics card information, CPU utilization, memory utilization, idle status, etc., and update the status table for server-side task scheduling strategy calculation.

[0165] 8. After receiving the task processing result, the transcoding server updates the subtask table. At the same time, the server will have a JOB to perform HOOK detection. When a main task is found to be completed, the business side system is called back. The callback method is specified by the transcoding template.

[0166] The transcoding client is deployed in a hybrid environment, including physical machines, Docker containers, cloud hosts, etc. in the formal and test environments. It can also be mixed with other services. The transcoding server controls the maximum resource utilization of the device through the policy module, greatly improving the overall resource utilization.

[0167] Due to the design of distributed locks, the transcoding server can implement HA deployment and can schedule tasks concurrently and safely among multiple instances. At the same time, when any transcoding client exits due to unexpected downtime or crash, the subtask can be re-activated by resetting the status of the subtask table, ensuring overall reliability and robustness.

[0168] In some embodiments, the transcoding task scheduling may include:

[0169] When the transcoding client pulls the transcoding subtask, it carries the device code parameter.

[0170] The transcoding server scheduling module queries all policy configurations corresponding to the device code.

[0171] The scheduling module parses and processes all scheduling strategies in turn.

[0172] For example, all scheduling strategies may refer to the first strategy to the fifth strategy in the aforementioned embodiment, which will not be described in detail here.

[0173] It can be seen from the above that the technical solution provided by the present invention has at least the following technical effects:

[0174] 1. Improve transcoding efficiency: A distributed, multi-business system-oriented, rich, production-level, large-scale video transcoding and metadata processing system is adopted to support more video metadata processing tasks, including transcoding, frame capture, voice transcription, text transcription, subtitle processing, etc. Distributed scheduling and execution are performed according to these types, and transcoding tasks can be assigned to multiple computing nodes for parallel processing, significantly improving transcoding speed and processing capabilities.

[0175] 2. Optimize resource utilization: Through a variety of scheduling strategies, such as time period control, machine CPU load and memory load scheduling strategies, multi-dimensional transcoding control functions are realized. At the same time, co-location of production environments is realized, and server resources are shared with other services to achieve true maximum resource utilization.

[0176] 3. Enhance system scalability: The distributed video transcoding method allows the system to be easily expanded according to demand, support more video transcoding tasks, and adapt to the growing video content and user needs.

[0177] 4. Reduce operating costs: By optimizing resource usage and improving transcoding efficiency, hardware investment and operation and maintenance costs can be reduced, thereby reducing overall operating costs.

[0178] 5. Improve system reliability and fault tolerance: The distributed architecture allows other nodes to take over tasks when a node fails, thereby improving the stability and reliability of the entire system.

[0179] 6. Flexible video format support: Leveraging the versatility of ffmpeg, this method can support a variety of video encoding formats and standards to meet the needs of different devices and platforms.

[0180] 7. Easy to maintain and upgrade: Distributed systems allow maintenance and upgrades of individual nodes without affecting the operation of the entire system, simplifying system management.

[0181] In some embodiments, a distributed video transcoding device is provided, which is applied to a transcoding server in a distributed system. The distributed system includes a transcoding server and multiple transcoding clients, such as Figure 6 As shown, the device comprises:

[0182] A receiving module 10 is used to receive a video transcoding task sent by a business system;

[0183] A decomposition module 20, used for decomposing the video transcoding task into a plurality of subtasks of the video to be transcoded, wherein different subtasks are used for processing different types of metadata of the video to be transcoded;

[0184] A storage module 30, used for storing multiple subtasks in a subtask table;

[0185] The scheduling module 40 is used to schedule the unscheduled subtasks in the subtask table to the transcoding clients that meet the scheduling strategy according to the current device status of each transcoding client and the scheduling strategy in the preset strategy table;

[0186] A generating module 50, configured to generate a transcoding result of the video to be transcoded after receiving the task execution results of all subtasks of the video to be transcoded;

[0187] The sending module 60 is used to send the transcoding result to the business system.

[0188] In some embodiments, the decomposition module 20 is used to:

[0189] Determine the target configuration corresponding to the business system from the preset transcoding template;

[0190] According to transcoding parameters of different types of metadata of the video to be transcoded in the target configuration, the video transcoding task is decomposed into multiple subtasks of the video to be transcoded.

[0191] In some embodiments, the scheduling module 40 is used to:

[0192] For any transcoding client, when receiving a task request from the transcoding client, judging whether the transcoding client satisfies all scheduling policies corresponding to the transcoding client in the preset policy table according to the current device state of the transcoding client, and obtaining a judgment result;

[0193] If the judgment result is yes, determine the unscheduled subtasks in the subtask table;

[0194] The transcoding client is used as the target transcoding client, and the unscheduled subtasks in the subtask table are scheduled to the target transcoding client.

[0195] In some embodiments, the scheduling strategy corresponding to the transcoding client includes at least one of the following:

[0196] A first policy is used to indicate a time period during which the transcoding client is allowed to execute tasks;

[0197] The second policy is used to indicate the CPU usage threshold when the transcoding client is allowed to execute the task;

[0198] The third policy is used to indicate a memory usage threshold when the transcoding client is allowed to execute a task;

[0199] A fourth strategy is used to instruct the transcoding client to execute a task of a preset priority;

[0200] A fifth policy includes a state value of a policy switch of the transcoding client, wherein the state value of the policy switch is used to indicate whether to perform task scheduling on the transcoding client.

[0201] In some embodiments, the storage module 30 is used to:

[0202] Each time a subtask is stored in the subtask table, a distributed lock is added to the subtask, wherein the distributed lock is used to prevent multiple transcoding clients from processing the same subtask at the same time;

[0203] The scheduling module 40 is used to:

[0204] The unscheduled subtasks in the subtask table are determined through distributed locks.

[0205] In some embodiments, the scheduling module 40 is further configured to:

[0206] In the case that there are unscheduled subtasks in the subtask table, if each transcoding client is in a state of executing tasks, the unscheduled subtasks in the subtask table are scheduled to the designated transcoding engine for processing according to the preset timed tasks.

[0207] In some embodiments, the subtask table is stored in a database, and the scheduling module is used to:

[0208] According to the current device status of each transcoding client and the scheduling policy in the preset policy table, the transcoding client that meets the scheduling policy is used as the target transcoding client, and the task information of the subtask assigned to the target transcoding client is updated in the subtask table. The updated task information of the subtask includes the allocation information of the subtask assigned to the target transcoding client; the subtask table is used for the target transcoding client to determine the subtask assigned to the target transcoding client after monitoring the allocation information of the subtask.

[0209] In some embodiments, the apparatus further comprises: a first monitoring module;

[0210] The first monitoring module is used for the transcoding server to monitor the execution status changes of each subtask in the subtask table to determine the execution status of each subtask.

[0211] In some embodiments, the apparatus further comprises: a second monitoring module;

[0212] The second monitoring module is used to monitor the running status of the transcoding client processing the subtask for each subtask of the video to be transcoded;

[0213] The scheduling module is also used to schedule the subtasks of the abnormally operating transcoding client to the normally operating transcoding client when it is monitored that the operating status of any transcoding client is abnormal.

[0214] It should be noted that those skilled in the art can understand that the method provided in the embodiment of the present invention can be executed alone or together with some methods in the embodiment of the present invention or some methods in related technologies.

[0215] It should be noted here that the description of the above data scanning device item is similar to the description of the above distributed video transcoding method item, and the description of the beneficial effects of the same method is not repeated. For technical details not disclosed in the data scanning device embodiment of the present invention, please refer to the description of the distributed video transcoding method embodiment of the present invention.

[0216] To achieve the above object, an embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, the device includes a processor 701 and a memory 703 connected to the processor 701 via a communication bus 702; wherein the memory 703 is used to store a computer program; the processor 701 is used to execute the computer program to implement the steps of the distributed video transcoding method described in any of the above schemes: receiving a video transcoding task sent by a business system; decomposing the video transcoding task into multiple subtasks of the video to be transcoded, and storing the multiple subtasks in a subtask table, wherein different subtasks are used to process different types of metadata of the video to be transcoded; according to the current device status of each of the transcoding clients and the scheduling strategy in the preset strategy table, scheduling the unscheduled subtasks in the subtask table to the transcoding client that meets the scheduling strategy; after receiving the task execution results of all subtasks of the video to be transcoded, generating the transcoding result of the video to be transcoded, and sending the transcoding result to the business system.

[0217] The processor 701 is used to execute the computer program to implement the following steps: determine the target configuration corresponding to the business system from a preset transcoding template; decompose the video transcoding task into multiple subtasks of the video to be transcoded according to the transcoding parameters of different types of metadata of the video to be transcoded in the target configuration.

[0218] The processor 701 is used to execute the computer program to implement the following steps: for any one of the transcoding clients, when receiving a task request from the transcoding client, judging whether the transcoding client satisfies all scheduling policies corresponding to the transcoding client in a preset policy table according to the current device state of the transcoding client, and obtaining a judgment result; when the judgment result is yes, determining the unscheduled subtasks in the subtask table; taking the transcoding client as the target transcoding client, scheduling the unscheduled subtasks in the subtask table to the target transcoding client.

[0219] The scheduling strategy corresponding to the transcoding client includes at least one of the following: a first strategy, used to indicate the time period in which the transcoding client is allowed to perform tasks; a second strategy, used to indicate the CPU usage threshold when the transcoding client is allowed to perform tasks; a third strategy, used to indicate the memory usage threshold when the transcoding client is allowed to perform tasks; a fourth strategy, used to instruct the transcoding client to perform tasks of preset priority; a fifth strategy, including a status value of a policy switch of the transcoding client, wherein the status value of the policy switch is used to indicate whether to schedule tasks for the transcoding client.

[0220] The processor 701 is used to execute the computer program to implement the following steps: for each subtask stored in the subtask table, a distributed lock is added to the subtask, wherein the distributed lock is used to prevent multiple transcoding clients from processing the same subtask at the same time; and the unscheduled subtasks in the subtask table are determined through the distributed lock.

[0221] Here, the processor 701 is used to execute the computer program to implement the following steps: when there are unscheduled subtasks in the subtask table, if each of the transcoding clients is in a state of executing tasks, the unscheduled subtasks in the subtask table are scheduled to the designated transcoding engine for processing according to the preset scheduled tasks.

[0222] The processor 701 is used to execute the computer program to implement the following steps: for each subtask of the video to be transcoded, the running status of the transcoding client that processes the subtask is monitored; when it is monitored that the running status of any one of the transcoding clients is abnormal, the subtask of the abnormally running transcoding client is scheduled to the normally running transcoding client.

[0223] Optionally, the processor 701 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Here, the program executed by the processor 701 may be stored in a memory 703 connected to the processor 701 via a communication bus 702, and the memory 703 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache.By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct RAM bus random access memory (DRRAM). The memory 703 described in the embodiment of the present invention is intended to include but is not limited to these and any other suitable types of memory 703. The memory 703 in the embodiment of the present invention is used to store various types of data to support the operation of the processor 701. Examples of such data include: any computer program operated by the processor 701, such as an operating system and application programs; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks.

[0224] In some embodiments, the communication bus 702 in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The communication bus 702 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0225] The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 701. The above processor 701 may be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located on the communication bus 702, and the processor 701 reads the information in the communication bus 702 and completes the steps of the above method in conjunction with its hardware. In some embodiments, the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0226] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0227] Yet another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions; after the computer-executable instructions are executed by a processor, the steps of the distributed video transcoding method provided in any one of the aforementioned embodiments can be implemented.

[0228] The computer-readable storage medium stores an executable program, and when the executable program is executed by the processor 701, the steps of the distributed video transcoding method applied to the endpoint device can be implemented. Figure 1 or Figure 2 or Figure 3 One or more of the methods shown.

[0229] In some embodiments, the computer storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0230] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0231] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.< / apps>

Claims

1. A distributed video transcoding method, characterized in that: The method is applied to a transcoding server in a distributed system, wherein the distributed system includes a transcoding server and multiple transcoding clients. The method includes: Receive video transcoding tasks sent by the business system; Decomposing the video transcoding task into a plurality of subtasks of the video to be transcoded, and storing the plurality of subtasks in a subtask table, wherein different subtasks are used to process different types of metadata of the video to be transcoded; According to the current device status of each of the transcoding clients and the scheduling policy in the preset policy table, the unscheduled subtasks in the subtask table are scheduled to the transcoding clients that meet the scheduling policy; After receiving the task execution results of all subtasks of the video to be transcoded, a transcoding result of the video to be transcoded is generated, and the transcoding result is sent to the business system.

2. The distributed video transcoding method according to claim 1, characterized in that: Decomposing the video transcoding task into multiple subtasks of the video to be transcoded, including: Determining a target configuration corresponding to the business system from a preset transcoding template; According to transcoding parameters of different types of metadata of the video to be transcoded in the target configuration, the video transcoding task is decomposed into a plurality of subtasks of the video to be transcoded.

3. The distributed video transcoding method according to claim 1, characterized in that: According to the current device status of each of the transcoding clients and the scheduling strategy in the preset strategy table, the unscheduled subtasks in the subtask table are scheduled to the transcoding clients that meet the scheduling strategy, including: For any of the transcoding clients, upon receiving a task request from the transcoding client, judging whether the transcoding client satisfies all scheduling policies corresponding to the transcoding client in the preset policy table according to the current device state of the transcoding client, and obtaining a judgment result; If the judgment result is yes, determining the unscheduled subtasks in the subtask table; The transcoding client is used as a target transcoding client, and the unscheduled subtasks in the subtask table are scheduled to the target transcoding client.

4. The distributed video transcoding method according to claim 3, characterized in that: The scheduling strategy corresponding to the transcoding client includes at least one of the following: A first policy, used to indicate a time period during which the transcoding client is allowed to execute a task; A second policy is used to indicate a CPU usage threshold when the transcoding client is allowed to execute a task; A third policy is used to indicate a memory usage threshold when the transcoding client is allowed to execute a task; A fourth strategy is used to instruct the transcoding client to execute a task of a preset priority; A fifth policy includes a state value of a policy switch of the transcoding client, wherein the state value of the policy switch is used to indicate whether to perform task scheduling on the transcoding client.

5. The distributed video transcoding method according to claim 3, characterized in that: The storing of the plurality of subtasks into the subtask table comprises: For each subtask stored in the subtask table, a distributed lock is added to the subtask, wherein the distributed lock is used to prevent multiple transcoding clients from processing the same subtask at the same time; The determining of the unscheduled subtasks in the subtask table includes: The unscheduled subtasks in the subtask table are determined by distributed locks.

6. The distributed video transcoding method according to any one of claims 1 to 5, characterized in that: The method further comprises: In the case where there are unscheduled subtasks in the subtask table, if each of the transcoding clients is in a state of executing tasks, the unscheduled subtasks in the subtask table are scheduled to a designated transcoding engine for processing according to a preset timed task.

7. The distributed video transcoding method according to any one of claims 1 to 5, characterized in that: The method further comprises: For each subtask of the video to be transcoded, monitoring the running status of the transcoding client processing the subtask; When it is monitored that the operating state of any one of the transcoding clients is abnormal, the subtask of the abnormally operating transcoding client is scheduled to the normally operating transcoding client.

8. A distributed video transcoding device, characterized in that: A transcoding server is applied to a distributed system, the distributed system includes a transcoding server and multiple transcoding clients, and the device includes: A receiving module, used to receive video transcoding tasks sent by the business system; A decomposition module, used for decomposing the video transcoding task into a plurality of subtasks of the video to be transcoded, wherein different subtasks are used for processing different types of metadata of the video to be transcoded; A storage module, used for storing the plurality of subtasks in a subtask table; A scheduling module, configured to schedule unscheduled subtasks in the subtask table to transcoding clients that meet the scheduling strategy according to the current device status of each transcoding client and the scheduling strategy in the preset strategy table; A generating module, configured to generate a transcoding result of the video to be transcoded after receiving the task execution results of all subtasks of the video to be transcoded; A sending module is used to send the transcoding result to the business system.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor executes the steps of the distributed video transcoding method according to any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions; after the computer-executable instructions are executed by the processor, the steps of the distributed video transcoding method according to any one of claims 1 to 7 can be implemented.

Images