A multimedia task scheduling method based on table lookup

Abstract

The present application discloses a multimedia task scheduling method based on a table lookup method. The method is applied to a multimedia board scheduling system, and the method includes: querying a suitable matching table for a layout corresponding to a chipset, finding all codec chip combinations that can meet the target conference, and querying a chip combination table to determine the codec chip combination scheme corresponding to each codec chip combination; according to the codec situation of each codec chip in each codec chip combination given in the chip codec path table, traversing all codec chips to determine the codec chip combination with the highest cost performance; according to the output layout of each codec chip given in the chip decoding output layout table, sending corresponding instructions to the codec chip in the codec chip combination with the highest cost performance. The present application does not need to rely on manual labor, and can automatically perform table lookup scheduling according to the actual conference layout of the target conference, dynamically allocate codec resources, improve resource allocation efficiency, and achieve full and reasonable use of resources.

Description

Technical Field

[0001] The present application relates to the field of multimedia processing technology, and in particular to a multimedia task scheduling method based on a table lookup method. Background Art

[0002] Multimedia task scheduling refers to the process of effectively allocating and managing audio, video, and data streams within multimedia systems. With the rapid development of network technology and multimedia applications, users are increasingly demanding high-quality, real-time audio and video communications. The core goal of multimedia task scheduling is to ensure the real-time transmission and processing of multimedia data by rationally allocating system resources. The design of scheduling algorithms not only prioritizes tasks but also considers multiple factors, such as network bandwidth, latency, and load balancing, to ensure smooth playback and efficient transmission of multimedia content.

[0003] Current multimedia task scheduling methods primarily rely on manual allocation, where system administrators manually assign multimedia tasks to different codec boards and processing units for resource scheduling. The inventors recognized that this method can meet basic task scheduling requirements in simple scenarios, but in complex scenarios, it requires significant time and effort, and manual allocation is particularly inefficient. Furthermore, manual allocation relies on the experience and judgment of the system administrator, which is often influenced by subjective factors and fails to fully utilize available codec resources, resulting in some codecs being idle, impacting overall system performance. Furthermore, it is difficult to ensure that it can adapt to different user needs and meeting layouts, resulting in poor flexibility. Summary of the Invention

[0004] The present application provides a multimedia task scheduling method based on a table lookup method, aiming to solve the technical problem of low efficiency of existing multimedia task scheduling methods.

[0005] A multimedia task scheduling method based on a table lookup method is applied to a multimedia board scheduling system. The multimedia board scheduling system includes a media board, and the media board includes multiple codec chips. Each codec chip is connected to an HDMI matrix and a switch. The method includes:

[0006] S1: The media board receives the conference information sent by the signaling board, analyzes it to obtain the target conference layout, and then searches the chip combination table for the corresponding chipset combination based on the target conference layout. It finds all codec chip combinations that can meet the target conference requirements and searches the chip combination table to determine the corresponding codec chip combination solution for each codec chip combination.

[0007] S2, according to the chip codec path table for each codec chip combination, traverse all codec chips, check the remaining codec capacity and HDMI resource usage of each codec chip, and determine the most cost-effective codec chip combination;

[0008] S3, according to the output layout of each codec chip given in the chip decoding output layout table, sends instructions to the codec chip responsible for decoding in the codec chip combination with the highest cost performance, informing it of the video stream to be decoded and the corresponding output layout, and sends instructions to the codec chip responsible for encoding, informing it of the encoding style and the pattern collected from the HDMI matrix.

[0009] In the above solution, optionally, before step S1, the method further includes:

[0010] List all group meeting layouts and create a layout ID table; the layout ID table records the layout ID corresponding to each group meeting layout and the corresponding layout split screen number;

[0011] According to different codec chip combinations, a chip combination table is established, wherein the chip combination table records the chip combination ID corresponding to each codec chip combination and the corresponding codec chip combination solution;

[0012] According to the parameters of each group layout in the layout ID table, determine all codec chip combinations that can meet each group layout from the chip combination table, and establish a layout corresponding chip combination matching table;

[0013] Establishing a chip codec path table, wherein the chip codec path table records the codec status of each codec chip in each group layout corresponding to each codec chip combination;

[0014] A chip decoding output layout table is established, wherein the chip decoding output layout table records the output layout of each codec chip corresponding to each codec chip combination for each group layout.

[0015] In the above solution, further optionally, step S1 further includes:

[0016] According to the group meeting layout of the target meeting, the layout ID table is queried, and the correctness is checked according to the corresponding layout split screen number.

[0017] In the above solution, optionally, the multiple codec chips include multiple high-performance codec chips and multiple standard codec chips, the high-performance codec chips can decode 8-channel 4K video streams, and the standard codec chips can decode 2-channel 4K video streams.

[0018] In the above scheme, further optionally, the media board is provided with a high-performance chip position and a standard chip position, and the high-performance chip position and the standard chip position are both provided with multiple slots; the multiple high-performance codec chips are respectively inserted into the multiple slots of the high-performance chip position, and the multiple standard codec chips are respectively inserted into the multiple slots of the standard chip position.

[0019] In the above scheme, further optionally, one of the high-performance codec chips serves as the main processor for executing steps S1-S3; the slot of the main processor is fixed, and the other high-performance codec chips are arbitrarily inserted into the remaining slots of the high-performance chip area; the multiple standard codec chips are arbitrarily inserted into the multiple slots of the standard chip area.

[0020] In the above solution, further optionally, each codec chip can read the slot number of the slot where it is located through the GPIO interface; after power-on, each codec chip configures its own intra-domain IP address according to the slot number of the slot where it is located.

[0021] In the above solution, further optionally, a TCP short link is used between the main processor and other codec chips.

[0022] In the above scheme, optionally, there are 13 codec chips in total; the HDMI matrix has 64 inputs and 26 outputs; the conference information includes participating terminals, media transceiver ports of terminals, media stream related information and group conference layout.

[0023] In the above solution, optionally, in the layout corresponding to the chipset matching table, all codec chip combinations corresponding to each group layout are arranged from high to low according to cost performance.

[0024] Compared with the prior art, this application has at least the following beneficial effects:

[0025] In the multimedia task scheduling method provided in the embodiment of the present application, after receiving the conference information, according to the group layout of the target conference, by querying the layout corresponding to the chipset matching table, chip combination table, chip codec path table, chip decoding output layout table and other tables, it is possible to automatically perform table lookup scheduling according to the actual conference layout of the target conference without relying on manual labor, and dynamically allocate codec resources, thereby improving resource allocation efficiency; the media board can receive the conference information from the signaling board in real time and quickly analyze the group layout of the target conference. This dynamic response mechanism enables the system to adjust resource configuration according to actual needs, rather than relying on fixed preset configurations, thereby reducing Waste of resources; relying on the table lookup method, the system can quickly find all codec chip combinations that meet the needs of the target meeting. This process greatly reduces the time for manual configuration and debugging, and improves scheduling efficiency; based on the chip combination obtained by the query, the system can clearly define the role and task of each codec chip to ensure that each chip can be fully utilized; when selecting a codec chip, the system will traverse the remaining codec capabilities and HDMI resource usage of all chips. This comprehensive resource evaluation can ensure that the system selects the most cost-effective chip combination, ensure that the selected combination can maximize the use of existing resources, achieve full and reasonable resource utilization, and improve resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of an application environment for a multimedia task scheduling method based on a table lookup method provided in one embodiment of the present application;

[0027] Figure 2 A flowchart of a multimedia task scheduling method based on a table lookup method is provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0029] In the description of the present application, unless otherwise specified, expressions such as “include”, “comprising”, “having”, etc. also mean “not limited to” (certain units, components, materials, steps, etc.).

[0030] In one embodiment, a multimedia task scheduling method based on a table lookup method is provided, which is applied to a multimedia board scheduling system, such as Figure 1 As shown, the multimedia board scheduling system includes a media board 1, which includes multiple codec chips 11, each of which is connected to an HDMI matrix 2 and a switch 3; Figure 2As shown, the method includes the following steps:

[0031] S1: The media board receives the conference information sent by the signaling board, analyzes it to obtain the target conference layout, and then searches the chip combination table for the corresponding chipset combination based on the target conference layout. It finds all codec chip combinations that can meet the target conference requirements and searches the chip combination table to determine the corresponding codec chip combination solution for each codec chip combination.

[0032] S2, according to the chip codec path table for each codec chip combination, traverse all codec chips, check the remaining codec capacity and HDMI resource usage of each codec chip, and determine the most cost-effective codec chip combination;

[0033] S3, according to the output layout of each codec chip given in the chip decoding output layout table, sends instructions to the codec chip responsible for decoding in the codec chip combination with the highest cost performance, informing it of the video stream to be decoded and the corresponding output layout, and sends instructions to the codec chip responsible for encoding, informing it of the encoding style and the pattern collected from the HDMI matrix.

[0034] Conference information includes participating terminals, their media transceiver ports, media stream information, and conference layout. Media stream information refers to conference formats, including resolution, bandwidth, frame rate, and protocol. The signaling board (primarily used to distribute conference parameters, including adding conference terminal members, conference layout mode, and conference formats (resolution, bandwidth, frame rate, etc.)) is used to send conference parameters.

[0035] A group conference layout refers to different visual presentation methods or screen configurations in a video conference. This layout determines how attendees' media streams are arranged and displayed on the display device, so that participants can clearly see each person or content in the conference.

[0036] Furthermore, before step S1, the method further includes:

[0037] List all group meeting layouts and create a layout ID table; the layout ID table records the layout ID corresponding to each group meeting layout and the corresponding number of layout split screens;

[0038] According to different codec chip combinations, a chip combination table is established, in which the chip combination ID and the corresponding codec chip combination scheme corresponding to each codec chip combination are recorded;

[0039] According to the parameters of each group layout in the layout ID table, determine all codec chip combinations that can meet each group layout from the chip combination table, and establish a layout corresponding chip combination matching table;

[0040] Establishing a chip codec path table, wherein the chip codec path table records the codec status of each codec chip in each group layout corresponding to each codec chip combination;

[0041] A chip decoding output layout table is established, wherein the chip decoding output layout table records the output layout of each codec chip corresponding to each codec chip combination for each group layout.

[0042] Among them, in the layout corresponding to the chipset matching table, each group will arrange all the corresponding codec chip combinations from high to low according to the cost performance.

[0043] In other words, the process of building each table is as follows:

[0044] 1: List all group layouts, form a layout ID table, and return the corresponding layout split screen number based on the ID. (Table 1)

[0045] The layout ID table can be found in Table 1 below. The left side of the curly brackets is the layout ID, and the number on the right is the number of split screens. When looking up the table, you can find the corresponding number of split screens based on the layout ID.

[0046] Table 1 Layout ID table

[0047] {1, 1}

[0048] {2, 2}

[0049] {3, 2}

[0050] {4, 3}

[0051] {5, 3}

[0052] {6, 3}

[0053] {7, 3}

[0054] {8, 4}

[0055] {9, 4}

[0056] {10, 4}

[0057] {11, 4)

[0058] {12, 4}

[0059] {13, 5)

[0060] {14, 5}

[0061] {15, 5}

[0062] {16, 5}

[0063] 2: Based on different board chip combinations, a chip combination table is formed. (Table 2)

[0064] Table 2 below shows the chip combination ID, indicating all chip combinations. For example, {0,0,1,cp_31} indicates a chip combination with ID number 0, which has 0 high-performance codec chips and 1 standard codec chip. Encoding is completed by the standard codec chip.

[0065] Table 2 Chip combination table

[0066] {0, 1, 1, CP_31}

[0067] {1, 1, 0, CP_36}

[0068] {2, 0, 2, CP_31}

[0069] {3, 1, 1, CP_31}

[0070] {4, 2, 0, CP_36}

[0071] {5, 0, 3, CP_31}

[0072] {6, 1, 2, CP_31}

[0073] {7, 2, 1, CP_31}

[0074] {8, 0, 4, CP_31}

[0075] {9, 1, 3, CP_31}

[0076] {10, 2, 2, CP_31}

[0077] {11, 3, 1, CP_31}

[0078] {12, 0, 5, CP_31}

[0079] {13, 1, 4, CP_31}

[0080] {14, 2, 3, CP_31}

[0081] {15, 3, 2, CP_31}

[0082] {16, 0, 6, CP_31}

[0083] {17, 1, 5, CP_31}

[0084] {18, 2, 4, CP_31}

[0085] {19, 4, 6, CP_31}

[0086] {20, 0, 7, CP_31}

[0087] {21, 1, 6, CP_31}

[0088] {22, 2, 5, CP_31}

[0089] {23, 4, 3, CP_31}

[0090] {24, 0, 8, CP_31}

[0091] {25, 1, 7, CP_31}

[0092] {26, 2, 6, CP_31}

[0093] {27, 4, 4, CP_31}

[0094] {28, 4, 7, CP_31}

[0095] 3: Based on the layout style, the number of participating members, and the resolution, find all board chip combinations that meet each layout. Among all combinations that meet the same layout, create a mapping table of layout IDs and layout combinations, sorted by cost-performance ratio from high to low. Each layout can have n chip combinations, and each combination can meet the layout. This is the layout-to-chipset matching table shown in Table 3 below. (Table 3)

[0096] Table 3 below shows the possible chip combinations for a given layout ID. Each combination ID corresponds to the chip combination in Table 2. For example, the fourth combination in Table 3, {4, {3, 6, 2, 4}}, indicates that when completing a conference with layout ID 4, there are four possible chip combinations: 3, 6, 2, and 4. 3 offers the best price / performance ratio. These 3, 6, 2, and 4 correspond to the chip combinations in Table 2.

[0097] Table 3 Layout corresponding chipset suitable matching table

[0098] {1, {1}}

[0099] {2, {1, 3, 2, 4}}

[0100] {3, {1, 3, 2, 4}}

[0101] {4, {3, 6, 2, 4}}

[0102] {5, {3, 6, 2, 4}}

[0103] {6, {3, 6, 2, 4}}

[0104] {7, {3, 6, 2, 4}}

[0105] {8, {3, 6, 9, 2, 4}}

[0106] {9, {3, 6, 9, 2, 4}}

[0107] {10, {3, 6, 9, 2, 4}}

[0108] {11, {3, 6, 9, 2, 4}}

[0109] {12, {3, 6, 9, 2, 4}}

[0110] {13, {6, 9, 13, 2, 4)}

[0111] {14, {6, 9, 13, 2, 4)}

[0112] {15, {6, 9, 13, 2, 4)}

[0113] {16, {6, 9, 13, 2, 4)}

[0114] {17, {6, 9, 13, 17, 2, 4, 7}}

[0115] {18, {6, 9, 13, 17, 2, 4, 7}}

[0116] {19, {6, 9, 13, 17, 2, 4, 7}}

[0117] {20, {6, 9, 13, 17, 2, 4, 7}}

[0118] {21, {6, 9, 13, 17, 2, 4, 7}}

[0119] {22, {9, 13, 17, 2, 4, 7}}

[0120] {23, {9, 13, 17, 2, 4, 7}}

[0121] {24, {9, 13, 17, 2, 4, 7}}

[0122] {25, {9, 13, 17, 2, 4, 7}}

[0123] 4: Based on the n chip combinations in each layout, a chip codec table is formed. The table contains the codec status of each board in each chip combination, including the number of code streams each board needs to decode and whether encoding is required. (Table 4)

[0124] Table 4 below shows the chip codec number table, which shows the optional codecs for the chip combination after the chip combination is determined in the previous step. For example, the 13th subscript in Table 4, {13, {{2, {{5}}}, {4, {{5}, {4, 1}}}}}, indicates that when the layout ID is 13, the chip combination can be 2 or 4. When chip combination 4 is selected, there are two codec solutions: one standard codec chip decodes 5 channels and the other encodes, or one standard codec chip decodes 4 channels and the other decodes 1 channel.

[0125] Table 4 Chip encoding and decoding path table

[0126] {1, {{1, {{1}}}}}

[0127] {2, {{1, {{2}}}, {2, {{2}}}, {3, {{1, 1}}}, {4, {{1, 1}}}}}

[0128] {3, {{1, {{2}}}, {2, {{2}}}, {3, {{1, 1}}}, {4, {{1, 1}}}}}

[0129] {4, {{2, {{3}}}, {3, {{2, 1}}}, {4, {{3}, {2, 1}, {1, 2)}}, {6, {{1, 1, 1}}}}

[0130] {5, {{2, {{3}}}, {3, {{2, 1}}}, {4, {{3}, {2, 1}, {1, 2)}}, {6, {{1, 1, 1}}}}

[0131] {6, {{2, {{3}}}, {3, {{2, 1}}}, {4, {{3}, {2, 1}, {1, 2)}}, {6, {{1, 1, 1}}}}

[0132] {7, {{2, {{3}}}, {3, {{2, 1}}}, {4, {{3}, {2, 1}, {1, 2)}}, {6, {{1, 1, 1}}}}

[0133] {8, {{2, {(4}}}, {3, {{2, 2}}}, {4, {{4}, {2, 2}}}, {6, {{2, 1, 1}}}, {9, ...}

[0134] {9, {{2, {(4}}}, {3, {{2, 2}}}, {4, {{4}, {2, 2}}}, {6, {{2, 1, 1}}}, {9, ...}

[0135] {10, {{2, {(4}}}, {3, {{2, 2}}}, {4, {{4}, {2, 2}}}, {6, {{2, 1, 1}}}, {9, ...}

[0136] {11, {{2, {(4}}}, {3, {{2, 2}}}, {4, {{4}, {2, 2}}}, {6, {{2, 1, 1}}}, {9, ...}

[0137] {12, {{2, {(4}}}, {3, {{2, 2}}}, {4, {{4}, {2, 2}}}, {6, {{2, 1, 1}}}, {9, ...}

[0138] {13, {{2, {{5})}, {4, {{5}, {4, 1}}}, {6, {{2, 2, 1)}}, {9, {{2, 1, 1, 1)}}, ...}

[0139] {14, {{2, {{5})}, {4, {{5}, {4, 1}}}, {6, {{2, 2, 1)}}, {9, {{2, 1, 1, 1)}}, ...}

[0140] {15, {{2, {{5})}, {4, {{5}, {4, 1}}}, {6, {{2, 2, 1)}}, {9, {{2, 1, 1, 1)}}, ...}

[0141] {16, {{2, {{5})}, {4, {{5}, {4, 1}}}, {6, {{2, 2, 1)}}, {9, {{2, 1, 1, 1)}}, ...}

[0142] 5: Based on the codec table, the number of codec paths for the combined chip is determined, but the chip does not yet know the layout style of the synthesized output after decoding. Therefore, another table is created to determine the output layout of each chip (HDMI output for the encoding chip to collect HDMI input) after determining the chip combination and the codec of each chip. (Table 5)

[0143] The chip decoding output layout table shown in Table 5 below is based on Table 4, and the decoding content of each chip is synthesized and output. The output layout format is the query style of Table 5.

[0144] Table 5 Chip decoding output layout table

[0145] {1, {{1, {{a}}}}}}

[0146] {2, {{1, {{a, a}}}, {2, {{x}}}, {3, {{a, b}}}, {4, {{x, a}}}}}

[0147] {3, {{1, {{a, a}}}, {2, {{x}}}, {3, {{a, b}}}, {4, {{x, a}}}}}

[0148] {4, {{2, {{x, x, x}}}, {3, {{a, a, b}}}, {4, {{x, x, x}, {x, x, a}, {x, a, a}}}}}

[0149] {5, {{2, {{x, x, x}}}, {3, {{a, a, b}}}, {4, {{x, x, x}, {x, x, a}, {x, a, a}}}}}

[0150] {6, {{2, {{x, x, x}}}, {3, {{a, a, b}}}, {4, {{x, x, x}, {x, x, a}, {x, a, a}}}}}

[0151] {7, {{2, {{x, x, x}}}, {3, {{a, a, b}}}, {4, {{x, x, x}, {x, x, a}, {x, a, a}}}}}

[0152] {8, {{2, {{x, x, x, x}}}, {3, {{a, a, b, b}}}, {4, {{x, x, x, x}, {x, x, a....

[0153] {9, {{2, {{x, x, x, x}}}, {3, {{a, a, b, b}}}, {4, {{x, x, x, x}, {a, x, x,

[0154] {10, {{2, {{x, x, x, x}}}, {3, {{a, a, b, b}}}, {4, {{x, x, x, x}, {x, a, x....

[0155] {11, {{2, {{x, x, x, x}}}, {3, {{a, a, b, b}}}, {4, {{x, x, x, x}, {x, x, x....

[0156] {12, {{2, {{x, x, x, x}}}, {3, {{a, a, b, b}}}, {4, {{x, x, x, x}, {a, x, x....

[0157] Furthermore, step S1 further includes:

[0158] According to the group meeting layout of the target meeting, the layout ID table is queried, and the correctness is checked according to the corresponding layout split screen number.

[0159] In other words, to implement this table lookup method, at least five tables are needed, namely, the layout ID table (Table 1), the chip combination table (Table 2), the layout corresponding chipset matching table (Table 3), the chip combination decoding path table (Table 4), and the chip decoding output layout table (Table 5).

[0160] Based on these five tables, once the conference layout and participant terminal parameters are determined, the table can be looked up based on the current media board usage to find the best solution for the conference.

[0161] Scheduling logic:

[0162] 1: After receiving conference-related signaling, the media board first analyzes the following points:

[0163] a: Participants, i.e., terminals (video conferencing terminals that send media streams to the media scheduling board and receive composite streams from the media board), as well as each terminal's media transceiver ports and media stream related information (resolution, protocol, etc.);

[0164] b: conference layout mode, conference ID;

[0165] 2: Based on the above statistical information, terminal members are added to the conference. After joining the conference, these terminals will send video streams to the media board and receive the synthesized streams from the media board.

[0166] 3: Based on the conference layout mode (layout ID), first query the layout mode conversion table (Table 1) to find the number of split screens based on the layout mode. Then perform a correctness check based on the number of split screens.

[0167] 4: Then query the optimal chip combination table (Table 3) based on the layout ID. Each layout ID corresponds to a minimum of four chip combinations. Each combination is a combination ID. The chip combination used by the combination ID can be found through the ID chip mapping table (Table 2), including how many high-performance chips (high-performance codec chips) and how many low-performance chips (standard codec chips) are used.

[0168] 5: According to the combination given in Table 4, traverse all boards, check the remaining encoding and decoding capabilities and HDMI resource usage of each board, and find the most cost-effective combination.

[0169] 6: Then, according to the chip combination, there is another mapping table (Table 5). According to the decoding situation of each board, TCP signaling is sent to inform the corresponding board of the code stream that needs to be decoded and the layout style that needs to be output on HDMI.

[0170] At the same time, inform the board that needs to be encoded, the style that needs to be encoded, and the image style collected from HDMI, and finally complete the encoding after cropping, decoding and synthesis.

[0171] After the conference parameters are distributed, the main scheduling board schedules media resources according to the table lookup method.

[0172] Furthermore, in the multimedia board scheduling system, the multiple codec chips include multiple high-performance codec chips and multiple standard codec chips.

[0173] Among them, there are 13 codec chips in total; 13 boards (5 high-performance codec chips plus 8 standard codec chips) constitute the media board. The high-performance codec chip, as a high-performance board, can decode 8-channel 4K video streams, and the standard codec chip can decode 2-channel 4K video streams.

[0174] Furthermore, the media board is provided with a high-performance chip area and a standard chip area, and both the high-performance chip area and the standard chip area are provided with multiple slots; multiple high-performance codec chips are respectively inserted into the multiple slots of the high-performance chip area, and multiple standard codec chips are respectively inserted into the multiple slots of the standard chip area.

[0175] One high-performance codec chip serves as the main processor and is used to execute steps S1-S3; the slot of the main processor is fixed, and other high-performance codec chips are arbitrarily inserted into the remaining slots of the high-performance chip area; multiple standard codec chips are arbitrarily inserted into multiple slots of the standard chip area.

[0176] Each codec chip can read its slot number through the GPIO interface; after power-up, each codec chip configures its own intra-domain IP address based on its slot number. TCP short links are used between the main processor and other codec chips.

[0177] In other words, the first high-performance codec chip serves as the main CPU, connected to the switch via eth0. eth1 serves as the external control port and network transceiver. Three additional network ports (also eth1) are also connected to the panel for out-of-band monitoring. There's also an HDMI matrix that connects to the HDMI outputs and inputs of 13 boards. There are 64 inputs (8x8) and 2x13 outputs, for a total of 26 channels. These are controlled by the master CPU or other devices (a total of four boards) via I2C, allowing for switching of control sources.

[0178] That is, in the multimedia card scheduling system provided by the embodiment of the present application, each card can read its own slot number (1-13) through GPIO. The main CPU slot number is fixed, and the high-performance codec chip and the standard codec chip are fixed in the same area and cannot be mixed. Cards in the same area can be inserted into any position at will.

[0179] After power-up, each card is assigned its own intra-domain IP address based on its slot number. This provides greater flexibility and reliability, avoids IP conflicts, and facilitates interaction with the master CPU. A short TCP link is used between the master CPU and the slave media cards to reduce duplicate heartbeats. There are three interfaces with the MCU signaling card: 1: merge, 2: add_input, and 3: remove_input.

[0180] The scheduling algorithm involved in the multimedia board scheduling system includes: counting all layout styles, and then determining how to complete each style (chip combination). For the same layout style, there can be n chip combinations, and each combination can meet the layout plan. Arrange from high to low cost performance, select several, and form a combination table. When the synthesis style is sent from the signaling board to the media board, first query the combination according to the table lookup method, and then see if the current resources can meet the combination: (1) If it can be met, cut the layout style into several pieces according to the current chip combination, distribute each terminal code stream participating in the layout to the selected combination chip, and select the chip that needs to complete the encoding (each combination will have an optimal chip as the main editing chip, specified in the table); (2) If it cannot be met, continue to select the suboptimal combination until a satisfactory combination is found and repeat step (1). After deciding on the main editing chip, configure the HDMI matrix through the I2C interface and switch the required HDMI input to the main editing chip.

[0181] In summary, the embodiment of the present application provides an algorithm system for multimedia board scheduling based on the table lookup method. The system uses multiple embedded codec boards to complete the media scheduling algorithm in a distributed manner and then sends it to each sub-board to complete the corresponding code stream decoding and synthetic encoding, thereby realizing the encoding and decoding of the media stream.

[0182] The algorithm system for multimedia board scheduling includes embedded codec boards, switch chips, HDMI matrix, etc.; the switch chips form a local area network by cascading embedded decoders; the media board mainly includes the media scheduling board and the media codec board, etc., which are interconnected through the network and HDMI matrix; the HDMI matrix is ​​used to output the media stream transmitted by the embedded distributed decoder and access the corresponding media board.

[0183] The beneficial effects of this application include:

[0184] 1) This card-based media scheduling system has rich encoding and decoding resources and can support up to 60 channels of 4K resolution decoding and 20 channels of encoding, allowing dozens of high-resolution conferences to be held online simultaneously.

[0185] 2) The card-type media scheduling system is flexible to use and can be equipped with different capability cards according to different user needs;

[0186] 3) The table lookup scheduling algorithm can automatically perform table lookup scheduling according to the conference layout type, fully utilizing the resources of each board and achieving full and reasonable resource utilization; the advantage of the scheduling method provided in this application is that the algorithm is flexible and can fully utilize the encoding and decoding resources.

[0187] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0188] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored, which involves all or part of the processes in the above-mentioned embodiment method.

[0189] In one embodiment, a computer program product is further provided, including a computer program / instruction, which implements all or part of the process in the above embodiment method when executed by a processor.

[0190] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A multimedia task scheduling method based on table lookup method, characterized in that: Applied to a multimedia board scheduling system, the multimedia board scheduling system includes a media board, the media board includes multiple codec chips, each codec chip is connected to an HDMI matrix and a switch; the method includes: S1: The media board receives the conference information sent by the signaling board, analyzes it to obtain the target conference layout, and then searches the chip combination table for the corresponding chipset combination based on the target conference layout. It finds all codec chip combinations that can meet the target conference requirements and searches the chip combination table to determine the corresponding codec chip combination solution for each codec chip combination. S2, according to the chip codec path table for each codec chip combination, traverse all codec chips, check the remaining codec capacity and HDMI resource usage of each codec chip, and determine the most cost-effective codec chip combination; S3, according to the output layout of each codec chip given in the chip decoding output layout table, sends instructions to the codec chip responsible for decoding in the codec chip combination with the highest cost performance, informing it of the video stream to be decoded and the corresponding output layout, and sends instructions to the codec chip responsible for encoding, informing it of the encoding style and the pattern collected from the HDMI matrix.

2. The multimedia task scheduling method based on table lookup method according to claim 1, characterized in that: Before step S1, the method further includes: List all group meeting layouts and create a layout ID table; the layout ID table records the layout ID corresponding to each group meeting layout and the corresponding layout split screen number; According to different codec chip combinations, a chip combination table is established, wherein the chip combination table records the chip combination ID corresponding to each codec chip combination and the corresponding codec chip combination solution; According to the parameters of each group layout in the layout ID table, determine all codec chip combinations that can meet each group layout from the chip combination table, and establish a layout corresponding chip combination matching table; Establishing a chip codec path table, wherein the chip codec path table records the codec status of each codec chip in each group layout corresponding to each codec chip combination; A chip decoding output layout table is established, wherein the chip decoding output layout table records the output layout of each codec chip corresponding to each codec chip combination for each group layout.

3. The multimedia task scheduling method based on table lookup method according to claim 2, characterized in that: Step S1 further includes: According to the group meeting layout of the target meeting, the layout ID table is queried, and the correctness is checked according to the corresponding layout split screen number.

4. The multimedia task scheduling method based on table lookup method according to claim 1, characterized in that: The multiple codec chips include multiple high-performance codec chips and multiple standard codec chips. The high-performance codec chips can decode 8-channel 4K video streams, and the standard codec chips can decode 2-channel 4K video streams.

5. The multimedia task scheduling method based on table lookup method according to claim 4 is characterized in that: The media board is provided with a high-performance chip area and a standard chip area, and the high-performance chip area and the standard chip area are both provided with multiple slots; the multiple high-performance codec chips are respectively inserted into the multiple slots of the high-performance chip area, and the multiple standard codec chips are respectively inserted into the multiple slots of the standard chip area.

6. The multimedia task scheduling method based on table lookup method according to claim 5, characterized in that: One of the high-performance codec chips serves as a main processor for executing steps S1-S3; the slot of the main processor is fixed, and the other high-performance codec chips are arbitrarily inserted into the remaining slots of the high-performance chip area; the multiple standard codec chips are arbitrarily inserted into the multiple slots of the standard chip area.

7. The multimedia task scheduling method based on table lookup method according to claim 5, characterized in that: Each codec chip can read the slot number of its slot through the GPIO interface; after power-on, each codec chip configures its own intra-domain IP address according to the slot number of its slot.

8. The multimedia task scheduling method based on table lookup method according to claim 6, characterized in that: A TCP short link is used between the main processor and other codec chips.

9. The multimedia task scheduling method based on table lookup method according to claim 1, characterized in that: There are 13 codec chips in total; the HDMI matrix has 64 inputs and 26 outputs; the conference information includes participating terminals, media transceiver ports of the terminals, media stream related information and group conference layout.

10. The multimedia task scheduling method based on table lookup method according to claim 1, characterized in that: In the table of suitable matching of chipsets corresponding to the layout, all codec chip combinations corresponding to each group are arranged from high to low according to the cost performance.

Images