Automatic testing device and method, program product and related equipment
By using the automated parsing and resource monitoring of the central scheduling module and the test case management module, the problem of low testing efficiency of heterogeneous computing cards was solved, and efficient and stable multi-format parallel testing was achieved, improving the testing efficiency and accuracy of heterogeneous computing cards.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HYGON INFORMATION TECH CO LTD
- Filing Date
- 2026-01-29
- Publication Date
- 2026-05-01
AI Technical Summary
Existing heterogeneous computing card video encoding and decoding testing solutions are inefficient, cannot adapt to heterogeneous computing cards from different manufacturers and models, rely on manual operation in the testing process, lack automation and unified scheduling, and have poor resource management, resulting in inaccurate test results and low efficiency.
It employs a central scheduling module and a test case management module to automatically parse test tasks, generate test cases adapted to different encoding formats, and optimize hardware resource allocation through a resource monitoring and scheduling module to decouple test logic from the execution environment, supporting multi-format parallel testing.
It improves the testing efficiency of heterogeneous computing cards, reduces time and manpower costs, ensures the accuracy and stability of test results, adapts to platforms with different encoding formats, supports multi-card parallel testing, and avoids resource contention and idle waste.
Smart Images
Figure CN121958005A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of chip testing, and in particular to an automatic testing device, method, program product and related equipment. Background Technology
[0002] With the rapid development of computer multimedia video, such as high-definition live video streaming, virtual reality (VR), and video-based intelligent security, the performance requirements for video encoding and decoding processing capabilities have increased dramatically. Video encoding and decoding, as a core technology in multimedia playback, involves complex algorithmic calculations, and its ever-increasing computational demands place extremely high demands on hardware processing capabilities. Traditional general-purpose processors (CPUs) face performance bottlenecks when processing high-resolution, high-frame-rate video streams, making it difficult to meet the dual requirements of real-time performance and energy efficiency.
[0003] To improve video encoding and decoding efficiency, heterogeneous computing architectures have emerged. Heterogeneous computing cards integrate multiple computing units to work collaboratively, fully leveraging the strengths of various processors in different video encoding and decoding tasks. Currently, many high-performance heterogeneous computing cards integrate dedicated video encoding and decoding engines, supporting multi-channel concurrency and multi-format compatibility hardware encoding and decoding capabilities.
[0004] However, current technical solutions for testing video codecs on heterogeneous computing cards require different test schemes designed by testers for different heterogeneous cards, resulting in low testing efficiency. Therefore, there is an urgent need for a functional testing scheme for audio and video codecs on heterogeneous computing cards to improve testing efficiency. Summary of the Invention
[0005] This invention provides an automated testing device, method, program product, and related equipment to improve the testing efficiency of heterogeneous computing cards.
[0006] In a first aspect, embodiments of the present invention provide an automatic testing device applied to video encoding and decoding testing of heterogeneous computing cards, comprising: Central scheduling module and test case management module; The central scheduling module is used to acquire and parse test tasks to obtain parsing results; based on the current encoding format included in the parsing results, it acquires a test case template matching the current encoding format from test case templates of different encoding formats stored in the test case management module; based on the current test data corresponding to the current encoding format included in the parsing results and the test case template, it generates a set of test cases that conform to the current encoding format; the set of test cases is used to provide to the target heterogeneous computing card so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain test results.
[0007] In a second aspect, embodiments of the present invention provide an automatic testing method, applied to the automatic testing apparatus as described in the first aspect, comprising: Obtain and parse the test task to get the parsing results; Based on the current encoding format included in the parsing results, a test case template matching the current encoding format is obtained from test case templates of different encoding formats; Based on the parsing results, including the current test data corresponding to the current encoding format and the test case template, a set of test cases conforming to the current encoding format is generated. The set of test cases is provided to the target heterogeneous computing card so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain test results.
[0008] Thirdly, embodiments of the present invention provide a computer program product, including a computer program that, when executed, implements the automatic testing method as described in the second aspect.
[0009] Fourthly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a program, and the processor calls the program stored in the memory to execute the automatic testing method as described in the second aspect.
[0010] Fifthly, embodiments of the present invention provide a storage medium storing a program that, when executed, implements the automatic testing method as described in the second aspect.
[0011] This invention provides an automated testing device for video encoding and decoding testing of heterogeneous computing cards, comprising: a central scheduling module and a test case management module; the central scheduling module is used to acquire and parse test tasks to obtain parsing results; based on the current encoding format included in the parsing results, it acquires a test case template matching the current encoding format from test case templates of different encoding formats stored in the test case management module; based on the current test data corresponding to the current encoding format included in the parsing results and the test case template, it generates a set of test cases conforming to the current encoding format; the set of test cases is used to provide to the target heterogeneous computing card, so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain test results.
[0012] The automatic testing device provided in this embodiment of the invention includes a central scheduling module and a test case management module. The central scheduling module first acquires and parses the test task to obtain the parsing result. The parsing result includes the current encoding format. Then, the central scheduling module can obtain a test case template that matches the current encoding format from the test case management module. After obtaining the test case template, the central scheduling module fills the current test data into the test case template to generate a set of test cases that conform to the current encoding format, thereby realizing the automatic generation of test cases for different encoding formats. At the same time, since a test task can contain multiple different encoding formats, the automatic testing device provided in this embodiment of the invention can acquire test case templates for different encoding formats and generate test cases that conform to different encoding formats. This allows it to adapt to different test platforms that support different encoding formats and can support the parallel execution of test cases with different encoding formats using the corresponding target heterogeneous computing cards, achieving mutual isolation and ultimately decoupling the test logic (a set of test cases that conform to different encoding formats) from the execution environment (target heterogeneous computing card). The management of test case templates with different encoding formats in the test case management module can improve the reusability of test case templates, reduce the cost of maintaining test case templates, and ensure the testing stability of heterogeneous computing cards. It can be seen that the technical solution provided by the embodiments of the present invention can reduce the time and manpower costs in the test case generation process and improve the testing efficiency of heterogeneous computing cards.
[0013] Furthermore, in the technical solution provided by the embodiments of the present invention, test case templates with different encoding formats are managed independently using the test case management module. The central scheduling module is only responsible for obtaining the test case template of the current encoding format and generating a set of test cases. The functions of the central scheduling module and the test case management module are set independently, thus achieving functional decoupling of different functional modules. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0015] Figure 1 This is a flowchart illustrating a video encoding and decoding test scheme for heterogeneous computing cards in related technologies. Figure 2 This is a schematic diagram of the structure of the automatic testing device provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the automatic testing method provided in an embodiment of the present invention.
[0016] Figure label: 21. User Interface; 22. Central Scheduling Module; 23. Test Case Management Module; 24. Log and Result Analysis Module; 25. Resource Monitoring and Scheduling Module; 26. Quality Assessment Module. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0018] With the rapid development of computer multimedia video, such as high-definition live video streaming, VR, and video-based intelligent security, the performance requirements for video encoding and decoding processing capabilities have increased dramatically. Video encoding and decoding, as a core technology in multimedia playback, involves complex algorithmic calculations, including encoding standards such as H.264, H.265, AV1, and AVS3. Its ever-increasing computational demands place extremely high demands on hardware processing capabilities. Traditional CPUs face performance bottlenecks when processing high-resolution, high-frame-rate video streams, making it difficult to meet the dual requirements of real-time performance and energy efficiency.
[0019] To improve video encoding and decoding efficiency, heterogeneous computing architectures have emerged. Heterogeneous computing cards integrate multiple computing units (such as CPUs, GPUs (Graphics Processing Units), FPGAs (Field-Programmable Gate Arrays), and dedicated codec accelerators) to work collaboratively, fully leveraging the strengths of each processor in different tasks. Currently, many high-performance heterogeneous computing cards integrate dedicated video codec engines, supporting multi-channel concurrency and multi-format compatible hardware encoding and decoding capabilities. However, due to the integration of various computing resources and codec hardware modules within heterogeneous computing cards, testing their video encoding and decoding functions faces numerous challenges. First, heterogeneous computing cards from different manufacturers and of different models differ in their encoding and decoding support capabilities (such as encoding format, resolution, bitrate, and multi-instance concurrency). Second, when heterogeneous computing cards with different architectures run simultaneously on an operating system, the compatibility between the hardware drivers adapted to different heterogeneous computing cards, their own firmware versions, and the operating system they run on is complex. If the compatibility is not properly matched, unexpected problems such as functional abnormalities and performance fluctuations will occur during testing, seriously affecting the accuracy and reliability of the test. Finally, the testing process needs to cover a large number of test cases, and non-automatic testing is inefficient and has a high frequency of problems.
[0020] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a video encoding and decoding test scheme for heterogeneous computing cards in related technologies.
[0021] Figure 1 The video encoding and decoding test scheme is illustrated using the manual video encoding and decoding test scheme based on the FFmpeg (Fast Forward Moving Picture Experts Group) command line as an example. Figure 1 The manual video encoding / decoding test scheme based on the FFmpeg command line described herein is widely used in the development phase of heterogeneous computing cards for preliminary testing of their hardware encoding / decoding capabilities. This test scheme typically involves engineers writing test scripts based on the FFmpeg command line to call underlying hardware acceleration interfaces to perform video encoding / decoding tests.
[0022] FFmpeg is an open-source, cross-platform audio and video processing framework that covers a range of audio and video processing functions, including recording, conversion, and streaming. It supports multiple audio and video formats and can run stably on various operating systems such as Windows, Mac OS, and Linux.
[0023] like Figure 1 As shown, the specific steps of the manual video encoding / decoding test scheme based on the FFmpeg command line mainly include: 1. Test Case Design: Manually design test cases based on the video encoding and decoding test requirements of heterogeneous computing cards. Test cases may include, for example, encoding format (e.g., H.264, H.265), resolution (e.g., 1080p, 4K, 8K), bitrate control mode (e.g., CBR (Constant Bit Rate), VBR), frame rate (e.g., 30fps, 60fps), and multi-instance concurrency (e.g., 1-channel, 4-channel, 8-channel parallel encoding).
[0024] 2. Script execution: By writing a Shell or Python script, the FFmpeg command line is called in a loop.
[0025] 3. Result Acquisition: The script records basic information such as command execution time, whether there are errors, and the size of the generated file. It can also call the ffprobe tool (a multimedia detection tool in the FFmpeg family) to analyze the format consistency of the output video.
[0026] 4. Manual Analysis: After the test is completed, engineers manually check the logs and compare the output video quality (e.g., by using VMAF (Video Multi-Method Assessment Fusion) and PSNR (Peak Signal-to-Noise Ratio) tools in a separate process) to determine whether the test has passed.
[0027] While FFmpeg command-line based testing solutions offer certain advantages in terms of flexibility, they suffer from the following significant problems in practical applications: 1. The testing process is highly dependent on manual intervention and has a low degree of automation: Each test case requires manually written scripts, and there is a lack of a unified scheduling mechanism for test tasks. When the number of test cases increases to hundreds or even thousands, omissions or erroneous executions are likely to occur, affecting the final test results.
[0028] 2. Disorganized test case management, lacking standardization and reusability: Because test scripts are written by different testers with varying parameter organization methods, flexible configuration or parameterized invocation is impossible. It is also difficult to delegate updates and maintenance to others.
[0029] 3. Lack of hardware resource scheduling capabilities, unable to adapt to heterogeneous environments: This solution cannot perceive the resource status of the underlying heterogeneous computing cards (such as GPU load and encoding / decoding engine usage), nor can it dynamically allocate tasks across multiple cards and instances. In multi-card parallel testing scenarios, resource contention or idle waste is likely to occur, affecting testing efficiency and result accuracy.
[0030] 4. Isolated results analysis and lack of a unified evaluation mechanism: Encoding / decoding quality evaluation usually requires additional tools, and the results are not automatically correlated with the original test cases. Error logs are scattered, lacking centralized log management and failure attribution analysis capabilities, resulting in time-consuming problem localization.
[0031] As can be seen from the above problems, the manual testing solution based on the FFmpeg command line has significant shortcomings in terms of automation, resource scheduling capabilities, test management standardization, result analysis integration, and system scalability. It is difficult to meet the video encoding and decoding testing needs of modern heterogeneous computing cards under high complexity, multiple scenarios, and high frequency iteration.
[0032] To address the aforementioned problems, this invention provides an automated testing device applied to video encoding and decoding testing of heterogeneous computing cards. Please refer to the schematic diagram of its structure. Figure 2 , Figure 2 This is a schematic diagram of the structure of the automatic testing device provided in an embodiment of the present invention.
[0033] like Figure 2 As shown, the automatic testing device includes: Central scheduling module 22 and test case management module 23; The central scheduling module 22 is used to acquire and parse test tasks to obtain parsing results; based on the current encoding format included in the parsing results, it acquires a test case template matching the current encoding format from the test case templates of different encoding formats stored in the test case management module 23; based on the current test data corresponding to the current encoding format included in the parsing results and the test case template, it generates a set of test cases that conform to the current encoding format; the set of test cases is used to provide to the target heterogeneous computing card so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain test results.
[0034] Please continue to refer to this. Figure 2 The central scheduling module 22 can acquire test tasks, for example, through the user interface 21. The user interface 21 can be implemented as a WEB (World Wide Web) interface, API (Application Programming Interface), or CLI (Command Line Interface).
[0035] The test tasks obtained from user interface 21 may include multiple test data using different encoding formats. Therefore, after parsing the test tasks, the parsing results may include test data using different encoding formats. In other words, the parsing phase can determine all the encoding formats to be tested and their corresponding test data, achieving a strong binding between encoding formats and test data. This ensures that when determining the current encoding format to be tested among multiple encoding formats, the current test data corresponding to the current encoding format can be obtained.
[0036] Please continue to refer to this. Figure 2 The format of the test tasks obtained from different user interfaces 21 is matched with the data interaction specification of the user interface 21. For example, the test tasks obtained from the WEB / API interface can be in JSON (JavaScript Object Notation) or XML (Extensible Markup Language) format; the test tasks obtained from the CLI interface can be in table format, key-value pair format, JSON format, or CSV (Comma-Separated Values) format.
[0037] Parsing a test task can include, for example, the following two steps: 1. The central scheduling module 22 converts the acquired test tasks into machine language.
[0038] 2. For machine language-based testing tasks, extract the current encoding format to be tested and the corresponding current test data to obtain the parsing results.
[0039] The test case templates stored in the test case management module 23 are structured test case description formats defined according to different encoding formats. Test data forming a set of test cases typically includes data information such as input sources, encoding parameters, expected outputs, and evaluation metrics. However, the description methods for this data differ in different encoding formats. Therefore, to adapt to test data in different encoding formats and automatically generate a set of test cases conforming to different encoding formats, this embodiment of the invention forms structured test case templates by using the description methods corresponding to test data in different encoding formats.
[0040] The target heterogeneous computing card refers to a heterogeneous computing card that can support the execution of a set of test cases generated by the central scheduling module 22 based on the test case template corresponding to the current encoding format and the current test data. The target heterogeneous computing card serves as the hardware execution environment for executing this set of test cases.
[0041] It is easy to understand that the specific type of the target heterogeneous computing card can be changed when the test case template and test data change.
[0042] As can be seen, the automatic testing device provided in this embodiment of the invention includes a central scheduling module 22 and a test case management module 23. The central scheduling module 22 first acquires and parses the test task to obtain the parsing result. The parsing result includes the current encoding format. Then, the central scheduling module 22 can obtain a test case template that matches the current encoding format from the test case management module 23. After the central scheduling module 22 obtains the test case template, it fills the current test data to be tested into the test case template, thereby generating a set of test cases that conform to the current encoding format, realizing the automatic generation of test cases with different encoding formats. At the same time, since a test task can contain multiple different encoding formats, the automatic testing device provided in this embodiment of the invention can acquire test case templates for different encoding formats and generate test cases that conform to different encoding formats. This allows it to adapt to different test platforms that are compatible with different encoding formats. It can also support the parallel execution of test cases with different encoding formats and use the corresponding target heterogeneous computing card for execution, achieving mutual isolation and ultimately decoupling the test logic (a set of test cases that conform to different encoding formats) from the execution environment (target heterogeneous computing card). The management of test case templates with different encoding formats in the test case management module 23 can improve the reusability of test case templates, reduce the cost of maintaining test case templates, and ensure the testing stability of heterogeneous computing cards. It can be seen that the technical solution provided by the embodiments of the present invention can reduce the time and manpower costs in the test case generation process and improve the testing efficiency of heterogeneous computing cards.
[0043] Furthermore, in the technical solution provided by the embodiments of the present invention, test case templates with different encoding formats are managed independently using the test case management module 23. The central scheduling module 22 is only responsible for obtaining the test case template of the current encoding format and generating a set of test cases. The functions of the central scheduling module 22 and the test case management module 23 are set independently, so the functions of different functional modules can also be decoupled.
[0044] In one embodiment, please refer to... Figure 2 The automatic testing device also includes: multiple heterogeneous computing cards and a resource monitoring and scheduling module 25; The resource monitoring and scheduling module 25 is used to monitor the hardware status information of each heterogeneous computing card and send the hardware status information to the central scheduling module 22. After generating a set of test cases conforming to the current encoding format based on the current test data corresponding to the current encoding format and the test case template included in the parsing result, the central scheduling module 22 is further configured to select a target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the set of test cases and in conjunction with the hardware status information; send the set of test cases to the resource monitoring and scheduling module 25, and notify the resource monitoring and scheduling module 25 to control the target heterogeneous computing card to execute the set of test cases.
[0045] Different heterogeneous computing cards have different performance, and their hardware status information is also different at the same time. Therefore, in order to execute a set of test cases better and more efficiently, the hardware status information of each heterogeneous computing card obtained from real-time monitoring can be combined to determine the appropriate heterogeneous computing card as the target heterogeneous computing card for executing the set of test cases.
[0046] The resource monitoring and scheduling module 25 in the automatic testing device provided in this embodiment of the invention can monitor the hardware status information of each heterogeneous computing card in real time. By acquiring the resources of the heterogeneous computing cards monitored by the resource monitoring and scheduling module 25, the central scheduling module 22 can improve the convenience of calling heterogeneous computing card resources (selecting the target heterogeneous computing card), thereby improving testing efficiency. Furthermore, by cooperating with the central scheduling module 22 in selecting heterogeneous computing cards, the real-time monitoring of the hardware status information of each heterogeneous computing card by the resource monitoring and scheduling module 25 enables unified perception and intelligent scheduling of heterogeneous computing card resources, achieving load balancing in a multi-heterogeneous computing card environment, avoiding resource contention or idling, and solving the problem mentioned in related technologies that FFmpeg command-line based testing schemes cannot dynamically adapt to hardware status. Therefore, the technical solution provided in this embodiment of the invention constructs a unified hardware abstraction layer to parameterize the capabilities and status of heterogeneous computing cards from different manufacturers and of different types, forming hardware status information, which is periodically collected by the resource monitoring and scheduling module 25 to achieve stable execution of a set of test cases.
[0047] Please continue to refer to this. Figure 2 ,exist Figure 2 In this context, heterogeneous computing cards may include, for example, NVIDIA GPUs (such as NVENC, NVIDIA Encoder, which is a hardware-based video encoder in NVIDIA GPUs), domestically produced heterogeneous computing cards, and CPU software encoding / decoding (such as Quick SYNC, Intel Quick Sync Video).
[0048] In one implementation, the central scheduling module 22 is used to select a target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the set of test cases and the hardware status information, including: Based on the current encoding format of the set of test cases and the hardware status information, the target heterogeneous computing card for executing the set of test cases is selected from multiple heterogeneous computing cards.
[0049] The current encoding format of a set of test cases can affect the adaptation cost of heterogeneous computing cards. Hardware status information can determine the available computing power / resources and other status information of heterogeneous computing cards. Therefore, the target heterogeneous computing card can be determined by combining the current encoding format and hardware status information.
[0050] For example, the central scheduling module 22 is used to select the target heterogeneous computing card for executing the set of test cases from multiple heterogeneous computing cards based on the current encoding format of the set of test cases and the hardware status information. This can be implemented in the following way: Based on the current encoding format of the set of test cases and the hardware status information, the weighted evaluation score of each heterogeneous computing card is determined; based on the weighted evaluation score of each heterogeneous computing card, the target heterogeneous computing card is selected.
[0051] The central scheduling module 22 performs weighted scoring calculations on heterogeneous computing cards based on test task requirements (such as the current encoding format) and multi-dimensional hardware status information such as the compatibility, load, and number of idle instances of the heterogeneous computing cards, so as to dynamically select the optimal heterogeneous computing card to execute (target heterogeneous computing card).
[0052] In one implementation, the hardware status information may include, for example, current load and idle codecs; The central scheduling module 22 is used to determine the weighted evaluation score of each heterogeneous computing card based on the current encoding format of the set of test cases and in conjunction with the hardware status information, including: For each heterogeneous computing card, perform the following steps: The ratio between the current load and the maximum allowable load threshold of the heterogeneous computing card is used as the first load value. The first load value is obtained by subtracting the first load value from the first value. The second load value is multiplied by the pre-configured load weight to obtain the load value. The first value can be, for example, 1.
[0053] The first load value is obtained by multiplying the ratio between the current load and the maximum allowable load threshold of the heterogeneous computing card with the load weight. The first load value can reflect the load utilization rate (load ratio) of the heterogeneous computing card. The second load value is obtained by subtracting the first load value (e.g., 1) from the first value. In contrast to the first load value, the second load value can reflect the idle rate of the heterogeneous computing card.
[0054] It is understood that the aforementioned first load value and second load value are only used to distinguish between the two. The first load value is used to reflect the load utilization rate of the heterogeneous computing card, and the second load value is used to reflect the idle rate of the heterogeneous computing card.
[0055] The current encoding format of the set of test cases is determined, and based on the current encoding format of the set of test cases and the encoding formats supported by the heterogeneous computing card, it is determined whether the heterogeneous computing card supports the current encoding format. The determination result is used as the encoding format value. The encoding format value is multiplied by the pre-configured encoding format weight to obtain the encoding format value. Specifically, for example, when the heterogeneous computing card can support the current encoding format of a set of test cases, the determination result is 1, and the encoder format value is 1; when the heterogeneous computing card does not support the current encoding format of each test case, the determination result is 0, and the encoder format value is 0.
[0056] The ratio of the idle encoder to the total number of encoders on the heterogeneous computing card is used as the encoder value; the encoder value is multiplied by the pre-configured encoder weight to obtain the encoder value. In other words, the encoder value reflects the idle coding rate of the heterogeneous computing card.
[0057] The sum of the load value, the encoding format value, and the encoder value is determined as the weighted evaluation score of the heterogeneous computing card.
[0058] In summary, if presented as a formula, the weighted evaluation score calculation formula for the heterogeneous computing card is as follows:
[0059] in: d i For the i-th heterogeneous computing card; Load i This represents the current load; MaxLoad i Maximum allowable load threshold; Compatibility i The result of determining whether the current encoding format of a set of test cases is supported; FreeInstances i TotalInstances represents the number of idle encoders.i This represents the total number of encoders.
[0060] The above-mentioned calculation of heterogeneous computing card scores is based on the idle rate and idle coding rate of the heterogeneous computing cards and their corresponding weights. It can select heterogeneous computing cards with higher idle rates and idle coding rates to execute the current set of test cases based on the real-time performance status of the heterogeneous computing cards. This improves the utilization rate of heterogeneous computing cards in the automated testing device, ensuring that the heterogeneous computing cards with higher idle rates and idle coding rates implement the set of test cases. This reduces the allocation of test tasks to heavily loaded heterogeneous computing cards, thereby extending their lifespan. Simultaneously, the heterogeneous computing card score is also based on whether the heterogeneous computing card can support the current encoding format of the set of test cases, determining the target heterogeneous computing card for executing the set of test cases, thus maximizing the matching degree between the set of test cases and the heterogeneous computing card. In summary, the above-mentioned weighted evaluation score calculation method can assign the most suitable heterogeneous computing card to execute a set of test cases, improving testing efficiency.
[0061] In other embodiments, the central scheduling module 22 is used to select a target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the current encoding format of the set of test cases and in conjunction with the hardware status information, including: Obtain a trained prediction model; the prediction model is constructed and trained based on historical test data formed by executing historical test tasks on various heterogeneous computing cards and a set of historical test cases in the historical test tasks. Based on the current encoding format of the set of test cases and the hardware status information, the predicted execution results of each heterogeneous computing card are output; based on the predicted execution results, the target heterogeneous computing card for executing the set of test cases is selected from multiple heterogeneous computing cards; the predicted execution results include execution efficiency and execution time.
[0062] The prediction model can perform test tasks by inputting feature maps to heterogeneous computing cards. The hardware status information of the heterogeneous computing card directly determines the upper limit of available resources and execution stability. Therefore, after training the prediction model, the real-time monitored hardware status information and the current encoding format are used as input to obtain the prediction execution results, so as to guide the selection of the best heterogeneous computing card (target heterogeneous computing card).
[0063] In some implementations, after generating a set of test cases conforming to the current encoding format based on the current test data corresponding to the current encoding format and the test case template included in the parsing result, the central scheduling module 22 is further used to select a target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the task priority corresponding to the test task and the current encoding format of the set of test cases, combined with the hardware status information.
[0064] The task priority of a test task can represent the weight of the time constraint for the test task execution, which can affect the determination result of the target heterogeneous computing card. For example, urgent test tasks with higher priority can be preferentially matched with heterogeneous computing cards with better hardware status information (low load, high availability resources) and adapted to the current encoding format as the target heterogeneous computing card. Non-urgent test tasks with lower priority can be flexibly scheduled when resources are scarce, ultimately achieving a dynamic optimal match between task priority and heterogeneous computing card resources.
[0065] Optionally, the central scheduling module 22 is used to select a target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the task priority corresponding to the test task and the current encoding format of the set of test cases, combined with the hardware status information, including: Determine the urgency level of different test tasks; Based on the determined urgency of the tasks, set the task priorities for different test tasks; The urgency of the test task is used to determine the corresponding task priority; Based on the determined task priority and the hardware status information, the target heterogeneous computing card for executing the set of test cases is selected from multiple heterogeneous computing cards.
[0066] In one embodiment, test tasks are prone to various anomalies during execution, the most common being instability of a single test case and large-scale task interruption. Instability of a single test case refers to test case failures not due to functional logic defects, but rather to external environmental factors, dependencies, or temporary issues. Examples include sudden high CPU / memory usage on the test environment server (e.g., other test tasks competing for resources), leading to interface timeouts and page loading failures; occasional unavailability of dependent third-party interfaces / services; and network fluctuations causing "request sending failures" in interface automation. Large-scale task interruption refers to test tasks being unexpectedly interrupted (e.g., machine shutdown, task crash). Examples include a sudden power outage or blue screen on the machine executing the test (e.g., during batch execution at night); large-scale regression testing (e.g., a version regression test with 1000+ test cases, 600 of which have been successfully executed, requiring several hours to restart after an interruption); and incorrect configuration changes causing task interruption (e.g., accidentally changing the test environment address midway, requiring correction and resumption from the breakpoint).
[0067] Therefore, during the execution of the test task, the central scheduling module 22 is also used to monitor the execution status of the test task, and to handle the abnormal status when the execution status of the test task is detected to be abnormal; wherein, the method for handling the abnormal status includes at least: failure retry and breakpoint continuation.
[0068] Retrying on failure refers to automatically re-executing a single test case 1 to N times after it fails. The actual execution result of the test case is determined by whether the retry was successful (rather than directly marking it as failed). Its core objective is to eliminate occasional failures caused by defects not inherent in the test case itself, thus avoiding misjudgments.
[0069] Resuming from a breakpoint refers to resuming execution of unfinished test cases from the point of interruption after a test task is unexpectedly interrupted (such as machine shutdown or task crash). Its core objective is to avoid repeatedly running successfully executed test cases, thus saving the overall time of large-scale test tasks.
[0070] The central scheduling module 22 monitors the execution status of test tasks during execution to facilitate timely handling of abnormal states. It supports handling abnormal states through retries on failure and resume execution from breakpoints, reducing the false positive rate of test task failures while saving execution time and improving execution efficiency. Both essentially aim to reduce ineffective execution; retries reduce the ineffective investigation of falsely identified defects, and resume execution from breakpoints reduces the unnecessary time spent on repeated executions, ultimately improving testing efficiency.
[0071] Based on the foregoing discussion, the test case management module 23 is also used to manage test case templates with different encoding formats; the management of test case templates with different encoding formats includes: category management and version control. For example, the category management can be: classifying test case templates based on different encoding formats and different versions of the test case template; the version control can be: after updating the version of the test case template, retaining the previous version of the test case template, and when an anomaly occurs in the current version of the test case template, reverting to the previous version of the test case template.
[0072] In one implementation, the test case template may be encoded in, for example, JSON or YAML (YAML Ain't Markup Language) format. For example, a JSON format test case template may look like this: { "test_id": ", / / test case ID "description": " ", / / The purpose of the test case "input_source": ", / / Path to the test case input file "encoder": ", / / encoder "parameters": { / / parameters "preset": ", / / Preset configuration "profile": ", / / Configuration file "level": " ", / / Maximum performance level for encoding "bitrate_mode": " ", / / Bitrate control mode "crf": / / Constant code rate factor }, "expected_output": " ", / / Expected output path "evaluation_metrics": [ ] / / Evaluation metrics } Based on the test case template in the JSON encoding format described above, the central scheduling module 22, in conjunction with the current test data included in the parsing results, can generate a set of test cases conforming to the JSON encoding format. A test case obtained can be as follows: { "test_id": "ENC-H265-001", / / Test case ID is ENC-H265-001 “description”: “H.265 encoding, CRF-23, 1080p”, / / Test target: encoding standard is H.265, bitrate control mode and parameter is 23, input original video resolution is 1080p; "input_source": " / videos / 1080p_60fps.yuv", “encoder”: “h265_nvenc”, / / The encoder for the test cases is the H.265 hardware encoder specific to NVIDIA GPUs.
[0073] "parameters": { "preset": "p4", / / NVIDIA NVENC encoder preset level; "profile": "main", / / Specifies the Main configuration file using the H.265 encoding standard; "level": "4.1", / / The upper limit of the encoding performance level is 4.1; “bitrate_mode”: “crf”, / / Bitrate control mode is constant quality factor (CRF); “crf”: 23 / / The specific quality parameter for CRF mode is 23. }, "expected_output": " / golden / h265_crf23.mp4", "evaluation_metrics": ["psnr", "ssim", "encoding_time"] / / Evaluation dimensions are: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Encoding Time (encoding_time) } To facilitate the execution of the test cases by heterogeneous computing cards, the central scheduling module 22 can also convert the test cases into test scripts and send the test scripts to the resource monitoring and scheduling module 25.
[0074] In order to preserve historical test scripts, the test case management module 23 can also be used to store historical test scripts and structure them to achieve centralized management and cross-project sharing of test assets.
[0075] To enable heterogeneous computing cards to execute test tasks more stably and reliably, in one embodiment, when the heterogeneous computing card executes test cases, test data can be loaded into video memory, and the performance status of video memory directly determines whether the data can be loaded normally.
[0076] Therefore, to further ensure that the video memory of the heterogeneous computing cards can support the execution of a set of test cases, the resource monitoring and scheduling module 25 is also used to collect video memory usage information of each heterogeneous computing card; when the video memory usage information indicates that the heterogeneous computing card cannot accept the test task, the resource monitoring and scheduling module 25 stops sending the hardware status information of the heterogeneous computing card that cannot accept the test task to the central scheduling module 22.
[0077] The resource monitoring and scheduling module 25 provided in this embodiment of the invention can not only monitor the hardware status information of each heterogeneous computing card, but also determine whether the video memory of the heterogeneous computing card can support the execution of the current set of test cases before sending the hardware status information to the central scheduling module 22. This allows the central scheduling module 22 to stop sending hardware status information of heterogeneous computing cards that cannot accept test tasks. This ensures that heterogeneous computing cards with high video memory usage are not included in the scope of heterogeneous computing cards used to calculate weighted evaluation scores. It also prevents heterogeneous computing cards with high video memory usage, low idle rate, and low idle coding rate from being assigned test tasks they cannot handle, thus improving the lifespan of heterogeneous computing cards. At the same time, it avoids slow test task execution caused by heterogeneous computing cards with high video memory usage, low idle rate, and low idle coding rate being unable to handle test tasks, thereby improving test efficiency. Furthermore, it avoids anomalies that occur during test task execution caused by heterogeneous computing cards with low idle rate and low idle coding rate being unable to handle test tasks.
[0078] In one embodiment, the heterogeneous computing card includes a hot-swappable device; the resource monitoring and scheduling module 25 is further configured to identify the hot-swappable device.
[0079] As is known, the core definition of a hot-swappable device is "hardware that supports the insertion, identification, use, or removal of a device without powering off while the system is powered on." Hot-swapping is a feature that some heterogeneous computing cards may possess, and heterogeneous computing cards with hot-swappable capabilities can be called hot-swappable devices.
[0080] Hot-swappable devices must simultaneously meet three key requirements: hardware design, protocol support, and system compatibility, to ensure safe and effective hot-swapping. In the functional testing of video encoding and decoding on heterogeneous computing cards, identifying hot-swappable devices can enhance testing value across four dimensions: efficiency, scenario coverage, risk control, and testing accuracy. This ensures that testing is both efficient and aligned with real-world application needs, while simultaneously reducing the risk of hardware damage.
[0081] Of course, the video encoding and decoding functional tests of heterogeneous computing cards also require evaluation of the test results. Therefore, please continue to refer to [the relevant documentation / reference]. Figure 2The automatic testing device further includes a quality assessment module 26; the quality assessment module 26 is used to generate an assessment report of the target heterogeneous computing card based on the expected results in the current test data, the output results of the target heterogeneous computing card, and the test results.
[0082] The quality assessment module 26 can realize automated and objective assessment of encoding and decoding quality. For example, it can integrate mainstream indicator calculation tools such as PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), and VMAF (Video Multimethod Assessment Fusion). After the test task is completed, it automatically compares the encoded output (output result of the target heterogeneous computing card) with the reference output (expected result) to generate a structured assessment report, providing quantitative basis for performance comparison and quality degradation analysis.
[0083] The quality assessment module 26 generates structured assessment reports based on mainstream indicator calculation tools in related technologies. It can adapt to test tasks corresponding to different test objectives of different heterogeneous computing cards, thereby improving test efficiency.
[0084] Please continue to refer to this. Figure 2 In order to preserve test resources, the aforementioned automatic testing device may also include, for example, a log and result analysis module 24; The log and result analysis module 24 is used to store the logs output by the central scheduling module 22, the test case management module 23, the resource monitoring and scheduling module 25, and the quality assessment module 26.
[0085] The log may include, for example, the test results of the central scheduling module 22 and the evaluation report of the quality assessment module 26.
[0086] Based on the functions of each module of the above-mentioned automatic testing device, it can be seen that the central scheduling module 22 provided in this embodiment of the invention can receive test tasks from the user interface 21 and parse the test tasks; obtain test case templates from the test case management module 23 and generate a set of test cases that conform to the current encoding format; send a set of test cases to the resource monitoring and scheduling module 25; the test case management module 23 can store test case templates and historical test scripts with different encoding formats; the resource monitoring and scheduling module 25 can monitor and obtain the hardware status information of each heterogeneous computing card in real time and transmit it to the central scheduling module 22, and can control the target heterogeneous computing card to execute a set of test cases; the quality evaluation module 26 can receive the output results of the target heterogeneous computing card, as well as the test results and evaluation test results; and the log and result analysis module 24 can collect the logs of each module.
[0087] As can be seen, the modules in the automatic testing device provided in this embodiment of the invention are functionally independent. In other words, the modules in the automatic testing device provided in this embodiment of the invention belong to a loosely coupled module architecture. This enables the automatic testing device provided in this embodiment of the invention to be updated based on newer heterogeneous accelerator cards, and only the modules that need to be updated can be updated, which can improve the adaptability of the automatic testing device provided in this embodiment of the invention and make it less likely to be obsolete.
[0088] In line with the loosely coupled structure described above, the various modules in the automatic testing device provided in this embodiment of the invention use AMQP (Advanced Message Queuing Protocol) for data transmission. Under the premise of ensuring that data is "not lost, not wrong, and not blocked," it achieves module decoupling, cross-environment compatibility, and asynchronous peak shaping. It is particularly suitable for scenarios such as video codec function testing of heterogeneous computing cards, which are characterized by "multiple devices, multiple languages, high reliability, and strong dynamism." It can solve the reliability problem of data transmission, reduce system coupling and operation and maintenance costs, and flexibly adapt to the needs of special scenarios such as hot-swappable devices and batch testing.
[0089] Based on the same inventive concept, this invention also provides an automatic testing method, the flowchart of which can be found in the provided text. Figure 3 , Figure 3 This is a schematic flowchart of an automatic testing method provided in an embodiment of the present invention. The automatic testing method is applied to the automatic testing apparatus as described in any of the foregoing embodiments.
[0090] like Figure 3 As shown, the method includes the following steps: Step S31: Obtain and parse the test task to get the parsing results.
[0091] Step S32: Based on the current encoding format included in the parsing result, obtain a test case template that matches the current encoding format from test case templates with different encoding formats.
[0092] Step S33: Based on the parsing results, including the current test data and test case template corresponding to the current encoding format, generate a set of test cases that conform to the current encoding format; the set of test cases is provided to the target heterogeneous computing card so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain the test results.
[0093] As can be seen, the automatic testing method provided in this embodiment of the invention acquires and parses test tasks based on an automatic testing device, obtaining parsing results. The parsing results include the current encoding format, which in turn allows the acquisition of test case templates matching the current encoding format. Then, the current test data is filled into the test case templates, thereby generating a set of test cases conforming to the current encoding format, achieving automated generation of test cases for different encoding formats. Furthermore, since a test task can contain multiple different encoding formats, the automatic testing method provided in this embodiment of the invention can acquire test case templates for different encoding formats and generate test cases conforming to different encoding formats. This allows it to adapt to different testing platforms with different encoding formats and supports parallel execution of test cases with different encoding formats using corresponding target heterogeneous computing cards, achieving mutual isolation and ultimately decoupling the test logic (a set of test cases conforming to different encoding formats) from the execution environment (target heterogeneous computing card). Therefore, the technical solution provided in this embodiment of the invention can reduce the time and labor costs in the test case generation process and improve the testing efficiency of heterogeneous computing cards.
[0094] Based on the same inventive concept, embodiments of the present invention also provide a computer program product, including a computer program, which, when executed, implements an automatic testing method as described in any of the foregoing embodiments.
[0095] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a program, and the processor calls the program stored in the memory to execute an automatic testing method as described in any of the foregoing embodiments.
[0096] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a program that, when executed, implements an automatic testing method as described in any of the foregoing embodiments.
[0097] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.
Claims
1. An automatic testing device, characterized in that, Video encoding and decoding tests applied to heterogeneous computing cards include: Central scheduling module and test case management module; The central scheduling module is used to acquire and parse test tasks to obtain parsing results; based on the current encoding format included in the parsing results, it acquires a test case template matching the current encoding format from test case templates of different encoding formats stored in the test case management module; based on the current test data corresponding to the current encoding format included in the parsing results and the test case template, it generates a set of test cases that conform to the current encoding format; the set of test cases is used to provide to the target heterogeneous computing card so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain test results.
2. The automatic testing device as described in claim 1, characterized in that, Also includes: Multiple heterogeneous computing cards and resource monitoring and scheduling modules; The resource monitoring and scheduling module is used to monitor the hardware status information of each heterogeneous computing card and send the hardware status information to the central scheduling module. After generating a set of test cases conforming to the current encoding format based on the current test data corresponding to the current encoding format and the test case template included in the parsing result, the central scheduling module is further used to select a target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the set of test cases and the hardware status information; send the set of test cases to the resource monitoring and scheduling module, and notify the resource monitoring and scheduling module to control the target heterogeneous computing card to execute the set of test cases.
3. The automatic testing device as described in claim 2, characterized in that, The central scheduling module is used to select the target heterogeneous computing card from multiple heterogeneous computing cards to execute the set of test cases based on the set of test cases and the hardware status information, including: Based on the current encoding format of the set of test cases and the hardware status information, the target heterogeneous computing card for executing the set of test cases is selected from multiple heterogeneous computing cards.
4. The automatic testing device as described in claim 3, characterized in that, The central scheduling module is used to select the target heterogeneous computing card for executing the set of test cases from multiple heterogeneous computing cards based on the current encoding format of the set of test cases and the hardware status information, including: Based on the current encoding format of the set of test cases and the hardware status information, the weighted evaluation score of each heterogeneous computing card is determined. The target heterogeneous computing card is selected based on the weighted evaluation scores of each heterogeneous computing card.
5. The automatic testing device as described in claim 4, characterized in that, The hardware status information includes: current load and idle codecs; The central scheduling module is used to determine the weighted evaluation score of each heterogeneous computing card based on the current encoding format of the set of test cases and the hardware status information, including: For each heterogeneous computing card, perform the following steps: The ratio between the current load and the maximum allowable load threshold of the heterogeneous computing card is used as the first load value. The first load value is obtained by subtracting the first load value from the first value. The second load value is multiplied by the pre-configured load weight to obtain the load value. The current encoding format of the set of test cases is determined, and based on the current encoding format of the set of test cases and the encoding formats supported by the heterogeneous computing card, it is determined whether the heterogeneous computing card supports the current encoding format. The determination result is used as the encoding format value. The encoding format value is multiplied by the pre-configured encoding format weight to obtain the encoding format value. The ratio of the idle encoder to the total number of encoders on the heterogeneous computing card is used as the encoder value; the encoder value is multiplied by the pre-configured encoder weight to obtain the encoder value. The sum of the load value, the encoding format value, and the encoder value is determined as the weighted evaluation score of the heterogeneous computing card.
6. The automatic testing device as described in claim 5, characterized in that, The determination result is 1 when the heterogeneous computing card supports the current encoding format of the set of test cases; the determination result is 0 when the heterogeneous computing card does not support the current encoding format of the set of test cases.
7. The automatic testing device as described in claim 2, characterized in that, The central scheduling module is used to select the target heterogeneous computing card for executing the set of test cases from multiple heterogeneous computing cards based on the current encoding format of the set of test cases and the hardware status information, including: Obtain a trained prediction model; the prediction model is constructed and trained based on historical test data formed by executing historical test tasks on various heterogeneous computing cards and a set of historical test cases in the historical test tasks. Based on the current encoding format of the set of test cases and the hardware status information, the predicted execution results of each heterogeneous computing card are output; based on the predicted execution results, the target heterogeneous computing card for executing the set of test cases is selected from multiple heterogeneous computing cards; the predicted execution results include execution efficiency and execution time.
8. The automatic testing device as described in claim 2, characterized in that, After generating a set of test cases conforming to the current encoding format based on the current test data corresponding to the current encoding format and the test case template included in the parsing results, the central scheduling module is further used to select the target heterogeneous computing card to execute the set of test cases from multiple heterogeneous computing cards based on the task priority corresponding to the test task and the current encoding format of the set of test cases, combined with the hardware status information.
9. The automatic testing device as described in claim 8, characterized in that, The central scheduling module is used to select the target heterogeneous computing card for executing the set of test cases from multiple heterogeneous computing cards based on the task priority corresponding to the test task and the current encoding format of the set of test cases, combined with the hardware status information, including: Determine the urgency level of different test tasks; Based on the determined urgency of the tasks, set the task priorities for different test tasks; The urgency of the test task is used to determine the corresponding task priority; Based on the determined task priority and the hardware status information, the target heterogeneous computing card for executing the set of test cases is selected from multiple heterogeneous computing cards.
10. The automatic testing device as described in claim 1, characterized in that, The central scheduling module is also used to monitor the execution status of the test task, and to handle the abnormal status when the execution status of the test task is detected to be abnormal; wherein, the method for handling the abnormal status includes at least: failure retry and breakpoint resumption. The test case management module is also used to manage test case templates with different encoding formats; the management of test case templates with different encoding formats includes: category management and version control.
11. The automatic testing device as described in claim 2, characterized in that, The resource monitoring and scheduling module is also used to collect the video memory usage information of each heterogeneous computing card; when the video memory usage information indicates that the heterogeneous computing card cannot accept the test task, the resource monitoring and scheduling module stops sending the hardware status information of the heterogeneous computing card that cannot accept the test task to the central scheduling module.
12. The automatic testing device as described in claim 11, characterized in that, The heterogeneous computing card includes a hot-swappable device; the resource monitoring and scheduling module is also used to identify the hot-swappable device. The automatic testing device further includes: The quality assessment module is used to generate an assessment report for the target heterogeneous computing card based on the expected results in the current test data, the output results of the target heterogeneous computing card, and the test results. The log and result analysis module is used to store the logs output by the central scheduling module, the test case management module, the resource monitoring and scheduling module, and the quality assessment module.
13. An automatic testing method, characterized in that, Applied to the automatic testing apparatus as described in any one of claims 1-12, comprising: Obtain and parse the test task to get the parsing results; Based on the current encoding format included in the parsing results, a test case template matching the current encoding format is obtained from test case templates of different encoding formats; Based on the parsing results, including the current test data corresponding to the current encoding format and the test case template, a set of test cases conforming to the current encoding format is generated. The set of test cases is provided to the target heterogeneous computing card so that the target heterogeneous computing card executes the set of test cases to test the video encoding and decoding function of the target heterogeneous computing card and obtain test results.
14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it implements the automatic testing method as described in claim 13.
15. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a program, and the processor calling the program stored in the memory to execute the automatic testing method as described in claim 13.
16. A storage medium, characterized in that, The storage medium stores a program that, when executed, implements the automatic testing method as described in claim 13.