Playback performance evaluation methods, systems, electronic devices and storage media

Abstract

This disclosure relates to a method, system, electronic device, and storage medium for evaluating playback performance. The method includes: sending test data to a multimedia device, wherein an evaluation device establishes a communication connection with the multimedia device; acquiring output information corresponding to the multimedia device; the output information includes information presented by the multimedia device when playing the test data; and the output information is used to evaluate the playback performance of the multimedia device. Through this disclosure, the evaluation device controls the multimedia device to play the test data and acquires the output information of the multimedia device when playing the test data. The rich output information is used to evaluate the multimedia playback performance of the multimedia device, and the evaluation result can reflect the true playback performance of the multimedia device.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to methods, systems, electronic devices and storage media for evaluating playback performance. Background Technology

[0002] Currently, there are various multimedia devices on the market, including mobile devices, televisions, set-top boxes, and computers. However, the playback performance of each device varies greatly, so it is necessary to evaluate the playback performance of multimedia devices. Existing methods for evaluating playback performance generally involve running the corresponding program directly on the multimedia device to collect hardware information and then assess the playback performance. However, this method obtains limited information and cannot evaluate the true playback performance of the multimedia device. Summary of the Invention

[0003] In view of this, this disclosure provides a method, apparatus, system, electronic device, storage medium, and computer program product for evaluating playback performance.

[0004] According to one aspect of this disclosure, a playback performance evaluation method is provided, applied to an evaluation device, the method comprising:

[0005] Test data is sent to a multimedia device, wherein a communication connection is established between the evaluation device and the multimedia device;

[0006] Obtain the output information corresponding to the multimedia device; the output information includes information presented by the multimedia device when playing the test data; wherein, the output information is used to evaluate the playback performance of the multimedia device.

[0007] In one possible implementation, the test data includes a first video test sequence; the output information includes information presented when the multimedia device plays the first video test sequence.

[0008] The information presented when the multimedia device plays the first video test sequence is used to evaluate the smoothness of the video playback on the multimedia device.

[0009] In one possible implementation, the first video test sequence includes multiple video frames corresponding to solid colors, wherein video frames corresponding to different solid colors are arranged alternately in the first video test sequence.

[0010] In one possible implementation, the information presented by the multimedia device when playing the first video test sequence includes a brightness value waveform.

[0011] The evaluation of the smoothness of video playback on the multimedia device includes:

[0012] Frame rate statistics are obtained by performing a Fourier transform on the brightness value waveform; the frame rate statistics represent the frequency of brightness value changes in the brightness value waveform.

[0013] Based on the frame rate statistics, the number of frames corresponding to each wavelength of the multiple solid colors is determined;

[0014] The rendering frame rate of the multimedia device is determined based on the number of frames corresponding to each pure color wavelength, in order to evaluate the smoothness of video playback on the multimedia device.

[0015] In one possible implementation, the test data includes RGB values ​​corresponding to at least one test point; the output information includes the chromaticity and / or brightness presented by the multimedia device when displaying based on the RGB values ​​corresponding to the at least one test point; wherein the chromaticity and / or brightness presented by the multimedia device when displaying based on the RGB values ​​corresponding to the at least one test point is used to evaluate the display color accuracy and / or display brightness mapping effect of the multimedia device.

[0016] In one possible implementation, the test data includes a second video test sequence; at least one frame in the second video test sequence contains a solid color region; the output information includes: the chroma and / or brightness of the solid color region when the multimedia device plays the second video test sequence; wherein the chroma and / or brightness of the solid color region when the multimedia device plays the second video test sequence is used to evaluate the high dynamic range (HDR) performance of the multimedia device.

[0017] In one possible implementation, the test data includes a third video test sequence and / or an audio test sequence; the output information includes user feedback on the playback effect when the multimedia device plays the third video test sequence and / or audio test sequence; wherein the user feedback on the playback effect is used to evaluate the playback performance of the multimedia device.

[0018] In one possible implementation, the method further includes:

[0019] Send a collection command to the color analyzer to instruct the color analyzer to collect the information presented by the multimedia device when playing the test data;

[0020] The step of obtaining the output information corresponding to the multimedia device includes:

[0021] The information collected by the color analyzer when the multimedia device plays the test data is received.

[0022] In one possible implementation, sending test data to the multimedia device includes:

[0023] At a first moment, a first instruction, a preset color value, and the test data are sent to the multimedia device. The first instruction is used to instruct the multimedia device to display the preset color and prepare to play according to the test data.

[0024] The step of obtaining the output information corresponding to the multimedia device includes:

[0025] The original presentation information of the multimedia device is acquired at fixed intervals, and a second instruction is sent to the multimedia device at a second moment. The second instruction is used to instruct the multimedia device to start playing according to the test data.

[0026] Based on the original presentation information and the preset color, valid messages are selected from the original presentation information as the output information corresponding to the multimedia device.

[0027] In one possible implementation, the method further includes:

[0028] Obtain the target display mode of the multimedia device; the test data corresponds to the target display mode.

[0029] According to another aspect of this disclosure, a playback performance evaluation apparatus is provided, applied to an evaluation device. The apparatus includes: a sending module for sending test data to a multimedia device, wherein a communication connection is established between the evaluation device and the multimedia device; and an acquisition module for acquiring output information corresponding to the multimedia device; the output information includes information presented by the multimedia device when playing the test data; wherein the output information is used to evaluate the playback performance of the multimedia device.

[0030] According to another aspect of this disclosure, a playback performance evaluation system is provided, the system comprising: an evaluation device and a multimedia device; wherein the evaluation device is used to send test data to the multimedia device; wherein the evaluation device and the multimedia device establish a communication connection; the multimedia device is used to receive the test data; and play the data according to the test data; the evaluation device is further used to acquire output information corresponding to the multimedia device; the output information includes information presented by the multimedia device when playing the data according to the test data; wherein the output information is used to evaluate the playback performance of the multimedia device.

[0031] In one possible implementation, the system further includes: a service center; the service center is used to update the test data and synchronize the updated test data to the evaluation device; the evaluation device is further used to evaluate the playback performance of the multimedia device based on the output information, and report the output information and the evaluation result of the playback performance of the multimedia device to the service center; the service center is further used to obtain the output information and the evaluation result of the playback performance of the multimedia device; and generate a multimedia playback performance report of the multimedia device based on the output information and the evaluation result of the playback performance of the multimedia device, and feed the multimedia playback performance report back to the evaluation device; the evaluation device is further used to receive the multimedia playback performance report.

[0032] In one possible implementation, the system further includes: a color analyzer; the evaluation device is also configured to send a collection command to the color analyzer; the color analyzer is configured to, in response to the collection command, collect information presented by the multimedia device when playing according to the test data, and feed back the collected information to the evaluation device.

[0033] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0034] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0035] According to another aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0036] In this embodiment, an evaluation device establishes a communication connection with a multimedia device. The evaluation device controls the multimedia device to play data based on test data and acquires the output information of the multimedia device during playback. This output information is used to evaluate the multimedia playback performance of the multimedia device. Since the output information includes information presented by the multimedia device during playback based on the test data, the evaluation result can reflect the true playback performance of the multimedia device. As an example, the evaluation device can use the output information to comprehensively evaluate the multimedia device's basic playback capabilities, decoding compatibility, actual rendering capabilities, audio playback capabilities, color accuracy, brightness mapping effects, HDR performance, post-processing capabilities, etc.

[0037] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0038] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0039] Figure 1 A schematic diagram of the architecture of an evaluation system according to an embodiment of the present disclosure is shown.

[0040] Figure 2 A schematic diagram of the components of an evaluation system according to an embodiment of the present disclosure is shown.

[0041] Figure 3 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0042] Figure 4 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0043] Figure 5 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0044] Figure 6 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0045] Figure 7 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0046] Figure 8 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0047] Figure 9A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown.

[0048] Figure 10 A structural diagram of a playback performance evaluation device according to an embodiment of the present disclosure is shown.

[0049] Figure 11 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0050] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0051] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this disclosure include a particular feature, structure, or characteristic described in connection with that embodiment. Therefore, the terms "exemplary," "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in various parts of this specification, do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized.

[0052] In this disclosure, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0053] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0054] Currently, the market offers a wide variety of multimedia devices, including mobile devices, televisions, set-top boxes (OTC), and computers. However, the playback performance of these devices varies significantly. For example, different devices support different playback formats, such as compatibility with maximum resolution, frame rate, bit depth, High Dynamic Range (HDR 10), High Dynamic Range Vivid (HDRVivid), and Dolby formats. Color reproduction also differs, with each device potentially offering multiple color modes, each with inconsistent color performance. Brightness also varies: peak brightness and brightness curve adjustments differ between devices. Different devices may apply different post-processing techniques to the multimedia playback, leading to variations in the final display effect. Furthermore, different devices have different physical resolutions and refresh rates, resulting in variations in screen performance across different physical resolutions. Therefore, a systematic evaluation of the playback performance of multimedia devices is needed. In related technologies, the hardware information of the multimedia device is collected by directly running the corresponding program on the multimedia device to be evaluated, and then the playback performance is evaluated. However, this method obtains limited information and cannot evaluate the true playback performance of the multimedia device.

[0055] To address the aforementioned technical problems, this disclosure provides a playback performance evaluation method (detailed description below). A communication connection is established between the evaluation device and the multimedia device to be evaluated. The evaluation device controls the multimedia device to play data based on test data and acquires the output information of the multimedia device during playback. This rich output information is used to evaluate the multimedia playback performance of the multimedia device. Since the output information includes information presented by the multimedia device during playback based on the test data, the evaluation result can reflect the true playback performance of the multimedia device. As an example, the evaluation device can use the output information to comprehensively evaluate the multimedia device's basic playback capabilities, decoding compatibility, actual rendering capabilities, audio playback capabilities, color accuracy (i.e., color precision), brightness mapping effect, HDR performance, post-processing capabilities, etc.

[0056] Figure 1 This diagram illustrates the architecture of an evaluation system according to an embodiment of the present disclosure, such as... Figure 1 As shown, the evaluation system may include: evaluation equipment 10 and multimedia equipment 20, and may also include service center 30 and color analyzer 40. Figure 2This diagram illustrates the structural components of an evaluation system according to an embodiment of the present disclosure. The evaluation system, or at least one component thereof, is used to perform the playback performance evaluation method provided in this embodiment of the present disclosure to evaluate the multimedia playback performance of the multimedia device 20.

[0057] Exemplarily, the evaluation device 10 can be a standalone device, integrated into other devices, or implemented through software or a combination of software and hardware. Exemplarily, the evaluation device 10 can be a device or system with data processing capabilities, or a component or chip within such devices or systems. For example, the evaluation device 10 can be a cloud server, desktop computer, portable computer, network server, PDA (personal digital assistant), mobile phone, tablet computer, wireless terminal device, embedded device, or other device with data processing capabilities, or a component or chip within such devices. As another example, the evaluation device 10 can also be a chip or processor with processing capabilities, and the evaluation device 10 can include multiple processors. Exemplarily, the evaluation device 10 can also have a display function, for example, it can be an electronic device with a display screen. Figure 2As shown, the evaluation device 10 may include: a first user interface (UI) module, a notification center module, a data synchronization module, a reporting module, a process control and management module, a data acquisition module, a color evaluation module, a brightness evaluation module, a performance and frame rate (smoothness) detection module, an HDR index evaluation module, a user test feedback module, a network communication module, a test data module, and a color analyzer module. The color analyzer module drives the color analyzer 40 to collect the chromaticity and brightness (i.e., xyY values ​​or xylv values) of the display screen of the multimedia device 20, where Y is the value of the tristimulus y, Y and lv both represent brightness, and x and y reflect the chromaticity characteristics of color). The test data module records the test data to be tested. The network communication module establishes a communication connection between the evaluation device 10 and the multimedia device 20. The evaluation device 10 sends instructions and test data to the multimedia device 20 through the network communication module, enabling the multimedia device 20 to play content based on the test data; the network communication module can also control the test progress. The data acquisition module collects, records, and stores data from the multimedia device 20, including chromaticity and luminance data collected by the color analyzer 40. The color evaluation module evaluates the display color accuracy of the multimedia device 20 based on the chromaticity data collected by the color analyzer 40. The luminance evaluation module evaluates the display luminance mapping effect of the multimedia device 20 based on the luminance data collected by the color analyzer 40. The performance and frame rate (smoothness) detection module queries information such as the bit depth and decoder supported by the multimedia device 20 and evaluates the smoothness of video playback. The HDR performance evaluation module evaluates the HDR performance of the multimedia device 20 based on the chromaticity and luminance data collected by the color analyzer 40. The user feedback module presents prompts to the user and obtains confirmation information entered by the user based on the prompts. Confirmation information may include: basic information about the multimedia device 20, and whether the playback effect of the multimedia device 20 is good. The process control and management module controls the overall testing process. It manages the communication connection between the evaluation device 10 and the multimedia device 20. It can also automatically control the test data for the multimedia device 20 based on its current display mode. Furthermore, it drives the color analyzer 40 via the network communication module to collect the chromaticity and brightness of the multimedia device 20 during playback based on the test data. Additionally, it can collect subjective feedback from users regarding the playback effect of the multimedia device 20. The notification center module receives notifications from the service center 30 and synchronizes the testing progress of the multimedia device 20 with the service center 30. Moreover, it allows interaction with the user through the first UI module, providing real-time feedback on the current test status so that the user can understand the testing progress and make adjustments accordingly.The data synchronization module is used to synchronize data with the service center 30. For example, this module can be used to update test data, upload evaluation results, and obtain approval results. The report module is used to analyze and evaluate data based on the multimedia device 20 and the color and brightness data collected by the color analyzer 40, and display the corresponding evaluation results through the first UI module. It can also obtain the approval report from the service center 30 on the playback performance of the multimedia device 20 and display it through the first UI module.

[0058] Exemplarily, the multimedia device 20 is a device to be evaluated that has video and / or audio playback functions. The multimedia device 20 may be configured with a display panel, which may include at least one of the following: light-emitting diode (LED), organic light-emitting diode (OLED), flexible light-emitting diode (FLED), liquid crystal display (LCD), mini light-emitting diode (miniled), active-matrix organic light-emitting diode (AMOLED), quantum dot light-emitting diodes (QLED), liquid crystal on silicon (LCoS), digital light processor (DLP), or any other type of display screen, etc. This disclosure does not limit the scope of the embodiments. As an example, the multimedia device 20 may be a mobile device, a television device, a set-top box device, or a computer device, etc. Figure 2As shown, the multimedia device 20 may include: a second UI module, a management module, a data communication module, a device capability acquisition module, a player module, and an image rendering module. The second UI module displays the interactive interface and user-operable control elements, allowing users to control the startup or shutdown of test applications, log in to account information, and select the display mode to be evaluated. The management module manages the scheduling between modules within the multimedia device 20 and manages different display modes. The data communication module supports communication between the multimedia device 20 and the evaluation device 10, receiving instructions and test data from the evaluation device 10 in real time, such as receiving color display instructions from the evaluation device 10 and displaying them through the image rendering module; it can also provide feedback to the evaluation device 10. The device capability acquisition module acquires basic playback information from the multimedia device 20, such as chip information, refresh rate, display width and height, decoder, and post-processing capabilities. The player module plays test data transmitted from the evaluation device 10, such as video and audio test sequences. The image rendering module supports rendering specified colors onto the display panel of the multimedia device 20 based on RGB values.

[0059] For example, the service center 30 can be a cloud server, and the service center 30 can establish a communication connection with the evaluation device 10 and / or the multimedia device 20. Figure 2 As shown, the service center 30 may include: a test data management module, a test equipment aggregation module, a device performance approval module, and a report generation module. The test data management module manages test data, supports updating test data, and can synchronize updated test data to the evaluation device 10 to ensure that the test data in the evaluation device 10 is updatable and controllable. The test equipment aggregation module receives the evaluation results reported by the evaluation device 10 after completing the evaluation of the multimedia device 20, as well as the data generated during the testing process of the multimedia device 20. It aggregates and manages the received data for subsequent approval and report generation. The device performance approval module, based on the data aggregated by the test equipment aggregation module and combined with manual approval, determines whether the playback performance of the multimedia device 20 meets the requirements and can also select the features supported by the multimedia device 20. The report generation module generates a corresponding report based on the playback performance of the multimedia device 20 and can provide this report to the evaluation device 10 and the user of the multimedia device 20.

[0060] For example, the color analyzer 40 has a collection range covering the display panel of the multimedia device 20. The color analyzer 40 establishes a communication connection with the evaluation device 10 and can quickly collect the chromaticity and brightness corresponding to the display screen of the multimedia device 20 under the drive of the evaluation device 10, and feed back the collected chromaticity and brightness to the evaluation device 10 in real time.

[0061] It should be noted that the evaluation system described in the embodiments of this disclosure is for the purpose of more clearly illustrating the technical solutions of the embodiments of this disclosure, and does not constitute a limitation on the technical solutions and evaluation system structure provided by the embodiments of this disclosure. As those skilled in the art will know, the technical solutions provided by the embodiments of this disclosure are also applicable to similar technical problems in the face of other similar or new scenarios.

[0062] The playback capability evaluation method provided in the embodiments of this disclosure will be described in detail below.

[0063] Figure 3 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown. Exemplarily, the method can be derived from the above... Figure 1 The evaluation device 10 shown performs an evaluation to assess the playback performance of the multimedia device 20, such as... Figure 3 As shown, the method may include the following steps:

[0064] Step 301: Send test data to the multimedia device.

[0065] In this process, a communication connection is established between the evaluation device and the multimedia device. For example, the evaluation device can activate a network communication module and transmit communication information to the multimedia device, thereby establishing a network connection with the multimedia device.

[0066] For example, the test data may include test images, video test sequences, audio test sequences, RGB values, specific brightness values, etc., and there are no limitations on this.

[0067] For example, the test image can be a solid color image, such as a red image (RGB(255, 0, 0)) or a green image (RGB(0, 255, 0)). Solid color images can display the same color across a wide range of output images, which is beneficial for collecting more comprehensive output information from multimedia devices.

[0068] For example, the audio test sequence can be an audio sequence in formats such as Dolby or DTS.

[0069] For example, a specific brightness value may include peak brightness, minimum brightness, and UI brightness.

[0070] For example, the RGB values ​​can include the RGB values ​​corresponding to at least one test point, where the value of R ranges from 0 to 255, the value of G ranges from 0 to 255, and the value of B ranges from 0 to 255. When the values ​​of R, G, and B are equal, they can be called grayscale values. The test point can be any node in the brightness range or any node in the color range, where each node in the brightness range represents a brightness level and each node in the color range represents a color. The RGB color values ​​and test point data can be adjusted according to actual requirements.

[0071] For example, the test video sequence may include video sequences of different specifications, such as video sequences with resolutions of 1080P, 2K, 4K, 8K, etc., video sequences with frame rates of 25 frames, 60 frames, 120 frames, etc., video sequences with bit depths of 8 bits, 10 bits, etc. For example, the evaluation device may also generate the above RGB values ​​into video sequences of corresponding specifications and formats according to factors such as the evaluation target and the formats supported by the multimedia device's player.

[0072] As an example, the test video sequence may include a first video test sequence; this first video test sequence includes multiple video frames corresponding to solid colors, wherein video frames corresponding to different solid colors are arranged alternately in the first video test sequence. The number and types of solid colors among the multiple solid colors can be set according to requirements and are not limited thereto. For example, the multiple solid colors can be two solid colors, red and green, and the first video test sequence consists of video frames corresponding to red and video frames corresponding to green arranged alternately.

[0073] As another example, the test video sequence may include a second video test sequence; the second video test sequence may be various Standard Dynamic Range (SDR) or HDR test sequences. For example, at least one frame in the second video test sequence contains a solid color region; wherein the size of the solid color region in the video frame and the number of video frames containing solid color regions can be set as needed, for example, all frames in the second video test sequence contain solid color regions.

[0074] As another example, the test video sequence can include a third video test sequence, which can be a normal color video sequence with resolutions such as 1080P, 2K, 4K, 8K, etc., a normal color video sequence with frame rates such as 25 frames, 60 frames, 120 frames, etc., a normal color video sequence with a bit depth of 8 bits, 10 bits, etc., and can also be various SDR video test sequences or HDR video test sequences.

[0075] In some scenarios, multimedia devices may have multiple display modes, which can also be called color display modes, such as normal mode, vivid mode, movie mode, sports mode, daylight outdoor mode, etc. In one possible implementation, the evaluation device can acquire the target display mode of the multimedia device. For example, a user can selectively choose the desired display mode for evaluation. For instance, multiple display mode options can be presented on the interactive page of the multimedia device or the interactive page of the evaluation device. In response to the user's selection of multiple display mode options, the target display mode to be evaluated is determined. For example, the evaluation device can also acquire all display modes of the multimedia device, iterate through each display mode, and sequentially use each display mode as the target display mode. For each display mode, steps 301-302 are executed to evaluate the playback performance of the multimedia device in each display mode.

[0076] The test data corresponds to the target mode. For example, the evaluation device can determine test data for testing the playback performance of the multimedia device under the target display mode. For example, the test data can differ for different display modes. For instance, a motion mode is suitable for watching video resources with large movements of people, such as sports games. The test data for this display mode can be a first video test sequence to evaluate the smoothness of video playback on the multimedia device in motion mode. As another example, a daylight outdoor mode is suitable for watching video resources with strong sunlight and ample light, such as beach scenery videos. The test data for this display mode can be a third video test sequence to evaluate the high dynamic range (HDR) performance of the multimedia device in daylight outdoor mode.

[0077] For example, the evaluation device can communicate with a service center and synchronize test data from the service center. The service center can update the test data and synchronize the updated test data to the evaluation device. After determining the test data corresponding to the target display mode, the evaluation device sends the test data to the multimedia device. Correspondingly, the multimedia device receives the test data and plays it, or plays the test data according to instructions from the evaluation device.

[0078] Step 302: Obtain the output information corresponding to the multimedia device; the output information includes the information presented by the multimedia device when playing the test data.

[0079] The "presented information" can be visual information presented by the multimedia device through the screen (e.g., the displayed image, color, brightness, etc.), auditory information presented by the multimedia device through its audio components (e.g., the played sound, the sound spectrum, etc.), or feedback information from the user after watching or listening to the video or audio played by the multimedia device. In one possible implementation, the evaluation device can send a collection command to the color analyzer, instructing the color analyzer to collect the information presented by the multimedia device when playing test data. Correspondingly, the color analyzer, in response to the collection command, collects the information presented by the multimedia device when playing the test data, such as color and brightness, and feeds the collected information back to the evaluation device. The evaluation device receives the information presented by the multimedia device when playing test data collected by the color analyzer. In this way, the evaluation device collects the information presented by the multimedia device when playing the test data in real time through the color analyzer, resulting in more accurate and reliable information.

[0080] As an example, in step 301 above, the evaluation device can send a third instruction and test data to the multimedia device. This third instruction instructs the multimedia device to pause playback at each preset time point while playing according to the test data (e.g., playing a video test sequence) until a continuation instruction is received. Upon receiving the third instruction and test data, the multimedia device, in response to the third instruction, pauses playback at each preset time point. Furthermore, after obtaining the information presented by the multimedia device at the current time point while playing according to the test data, the evaluation device sends a continuation instruction to the multimedia device. The multimedia device, in response to this instruction, continues playback according to the test data and pauses playback at the next preset time point. Thus, by pausing playback at each preset time point, the evaluation device can synchronously obtain the information presented by the multimedia device at each time point while playing according to the test data, thereby achieving progress synchronization between the evaluation device obtaining the information presented by the multimedia device while playing according to the test data and the multimedia device playing according to the test data.

[0081] As another example, in step 301 above, the evaluation device can send a first instruction, a preset color value, and test data to the multimedia device at a first moment. The first instruction instructs the multimedia device to display the preset color and prepare for playback based on the test data. Upon receiving the first instruction, the preset color value, and the test data, the multimedia device, in response to the first instruction, enters a waiting state to prepare for playback and displays the preset color on its display panel. Then, the evaluation device can acquire the original presentation information of the multimedia device at fixed intervals. For example, it can control a color analyzer to start collecting the chromaticity and / or brightness of the multimedia device's display at fixed intervals. At a second moment, the evaluation device sends a second instruction to the multimedia device, instructing it to start playback. Upon receiving the second instruction, the multimedia device removes the preset color from its display panel (e.g., removes the layer displaying the preset color) and transitions from the waiting state to the playback state, playing the video based on the test data. Furthermore, the evaluation device can filter out valid messages from the original presentation information and preset colors as the corresponding output information of the multimedia device. It is understood that the original presentation information includes information about the multimedia device displaying the preset color and information about the multimedia device playing according to test data. For example, the evaluation device can select the information with the largest deviation from the preset color from the original presentation information. The acquisition time point corresponding to this information is the time point when the color analyzer begins acquiring information about the multimedia device playing according to test data. Therefore, information after this acquisition time point in the original presentation information can be considered valid information, i.e., the output information of the multimedia device. For example, the preset color can be green RGB(0, 255, 0). The evaluation device can determine the brightness value with the largest deviation from RGB(0, 255, 0) based on the brightness of the multimedia device's display screen acquired by the color analyzer, and use the time point corresponding to this brightness value as the starting time point for the color analyzer to acquire the brightness of the multimedia device playing according to test data. This allows for accurate filtering of the brightness of the multimedia device playing according to test data from all the brightness values ​​acquired by the color analyzer. The preset colors and fixed intervals can be set according to requirements and the type of test data, and are not limited thereto. This allows the multimedia device to play data without pausing, avoiding potential adjustments to display brightness within the multimedia device when it pauses. This achieves smoother progress synchronization between the information presented by the evaluation device during playback based on the test data and the actual playback process.

[0082] The output information is used to evaluate the playback performance of the multimedia device; the playback performance may include video display performance and / or audio playback performance. For example, the output information may include one or more of the following: information presented when the multimedia device plays the first video test sequence, chroma and / or brightness presented when the multimedia device displays the RGB values ​​corresponding to the at least one test point, chroma and / or brightness presented in the solid color area when the multimedia device plays the second video test sequence, user feedback information regarding the playback effect when the multimedia device plays the third video test sequence and / or audio test sequence, and brightness presented when the multimedia device displays the specific brightness value. The information presented when the multimedia device plays the first video test sequence can be used to evaluate the smoothness of video playback on the multimedia device. The chroma and / or brightness presented when the multimedia device displays the RGB values ​​corresponding to the at least one test point can be used to evaluate the display color accuracy and / or display brightness mapping effect of the multimedia device. The chroma and / or brightness presented in the solid color area when the multimedia device plays the second video test sequence can be used to evaluate the high dynamic range (HDR) performance of the multimedia device. The brightness displayed by a multimedia device when showing a specific brightness value can be used to evaluate the quantification accuracy of the display brightness and the smoothness of brightness changes.

[0083] Furthermore, both the evaluation equipment and / or service center can assess the playback performance of the multimedia device based on its output information.

[0084] Scenario 1: The evaluation device can evaluate the playback performance of the multimedia device based on the output information of the multimedia device.

[0085] For example, the evaluation device can assess the smoothness of video playback on the multimedia device based on the information presented when the multimedia device plays the first video test sequence. It can also evaluate the display color accuracy and / or display brightness mapping effect of the multimedia device based on the chromaticity and / or brightness presented when the multimedia device displays the RGB values ​​corresponding to at least one test point. Furthermore, it can evaluate the high dynamic range (HDR) performance of the multimedia device based on the chromaticity and / or brightness presented in the pure color areas when the multimedia device plays the second video test sequence. Finally, it can evaluate the quantization accuracy of the display brightness and the smoothness of brightness changes based on the brightness presented when the multimedia device displays a specific brightness value. Thus, by analyzing and evaluating the acquired objective data, it is possible to determine whether the multimedia device supports various playback performance aspects and the quality of its support.

[0086] For example, the evaluation device can also assess the playback performance of the multimedia device based on user feedback regarding the playback effect. In this way, by combining subjective human feedback on the playback effect of the multimedia device, the playback performance is evaluated, thus providing a comprehensive assessment of whether the playback performance of the multimedia device meets the expected level, both objectively and subjectively.

[0087] As an example, the evaluation device can also obtain basic playback information of the multimedia device, such as chip information, refresh rate, display width and height, bit depth, decoder, and post-processing capabilities (such as frame interpolation, sharpening, and color enhancement). For instance, the evaluation device can query information such as the bit depth and decoder supported by the multimedia device through the Application Programming Interface (API), or it can obtain basic playback information from user feedback. For example, it can send prompts to the user, who can then view relevant information about the multimedia device and provide feedback to the evaluation device. This basic playback information reflects the performance of the multimedia device in terms of its basic playback capabilities, decoding compatibility, and post-processing capabilities. Therefore, the evaluation device can perform a more comprehensive evaluation of the multimedia device's playback performance based on this basic playback information and the aforementioned output information.

[0088] In scenario two, the evaluation device can also report the output information of the multimedia device to the service center. After receiving the output information, the service center evaluates the playback performance of the multimedia device based on this information. The possible implementation methods for the service center's evaluation can be found in the description of the evaluation device's assessment of the multimedia device's playback performance, and will not be repeated here.

[0089] Scenario 3: The evaluation device can also report the output information of the multimedia device and the results of its playback performance evaluation to the service center. After receiving the output information and the evaluation results, the service center can summarize and analyze the data to generate a multimedia playback performance report for the multimedia device. This report is then sent back to the evaluation device and / or the multimedia device itself. Upon receiving the report, the evaluation device and / or the multimedia device can display it to the user for review and to provide a reference for optimizing the device's playback performance. Furthermore, the service center can also certify the playback performance of the multimedia device and generate a certification report.

[0090] In this embodiment, a communication connection is established between the evaluation device and the multimedia device. The evaluation device controls the multimedia device to play data based on test data and acquires the output information of the multimedia device during playback. This rich output information is used to evaluate the multimedia playback performance of the multimedia device. Since the output information includes information presented by the multimedia device during playback based on the test data, the evaluation result can reflect the true playback performance of the multimedia device. As an example, the evaluation device can use the output information to comprehensively evaluate the multimedia device's basic playback capabilities, decoding compatibility, actual rendering capabilities, audio playback capabilities, color accuracy, brightness mapping effects, HDR performance, and post-processing capabilities.

[0091] The following section uses test data as the first video test sequence as an example to explain the specific process of evaluating the playback performance of multimedia devices in the above scenario one.

[0092] Figure 4 A flowchart illustrating a playback performance evaluation method according to an embodiment of this disclosure is shown. Figure 4 As shown, the method may include the following steps:

[0093] Step 401: Send the first video test sequence to the multimedia device.

[0094] The first video test sequence, also known as the frame rate test sequence, is used to test the rendering frame rate of multimedia devices.

[0095] In one possible implementation, the first video test sequence includes multiple video frames corresponding to solid colors, wherein video frames corresponding to different solid colors are arranged alternately in the first video test sequence. For example, the first video test sequence consists of alternating arrangements of video frames corresponding to red and video frames corresponding to green, that is, in the first video test sequence, pure red images and pure green images are displayed alternately.

[0096] For example, a target display mode of a multimedia device can be obtained; and a first video test sequence corresponding to the target display mode can be determined.

[0097] Step 402: Obtain the information presented when the multimedia device plays the first video test sequence.

[0098] In one possible implementation, the information presented by the multimedia device when playing the first video test sequence includes a brightness value waveform. For example, the evaluation device can control the color analyzer to collect the brightness presented by the multimedia device when playing the first video test sequence. For instance, when the multimedia device plays the first video test sequence, the color analyzer can continuously and rapidly collect the actual brightness displayed by the multimedia device, such as collecting 3,000 brightness values ​​within 1 second. Then, based on the collected actual brightness values, a brightness value waveform (i.e., a waveform showing the change of brightness over time) is generated in real time, and the evaluation device can obtain the brightness value waveform generated by the color analyzer.

[0099] Step 403: Evaluate the smoothness of video playback on the multimedia device based on the information presented when the multimedia device plays the first video test sequence.

[0100] In one possible implementation, the evaluation device can obtain frame rate statistics by performing a Fourier transform on the aforementioned brightness value waveform. The frame rate statistics represent the frequency of brightness value changes in the brightness value waveform. Based on the frame rate statistics, the number of frames corresponding to each wavelength of the plurality of pure colors is determined. According to the number of frames corresponding to each pure color wavelength, the rendering frame rate of the multimedia device is determined to evaluate the smoothness of video playback on the multimedia device. A higher rendering frame rate indicates higher smoothness of video playback on the multimedia device, and vice versa. For example, the number of times the brightness of each pure color appears in each cycle can be determined based on the brightness differences of different pure colors and the frame rate statistics, i.e., the number of frames corresponding to each pure color wavelength in each cycle, thereby determining the rendering frame rate of the multimedia device. For example, after determining the number of frames corresponding to each pure color wavelength within a period, effective period filtering can be performed, that is, determining whether each pure color appears alternately in adjacent consecutive frames within the period. If they all appear, the period is considered valid; otherwise, the period is considered to have dropped frames. Thus, the rendering frame rate of the multimedia device can be determined by the number of frames within the effective period.

[0101] In one possible implementation, the first video test sequence consists of alternating red and green video frames. The evaluation device can obtain the number of peaks and troughs in each period from the brightness value waveform diagram, which are respectively used as the number of red and green video frames in each period. By statistics, the number of red and green frames in the period can be determined, that is, the number of valid frames in the period can be determined, thereby obtaining the rendering frame rate of the multimedia device.

[0102] In this embodiment, the smoothness of video playback on the multimedia device is evaluated based on information presented when the multimedia device plays a first video test sequence. Smoothness reflects the actual rendering capability of the multimedia device. As an example, the first video test sequence consists of alternating red and green video frames. By analyzing the brightness waveform collected by a color analyzer, the actual on-screen rendering frame rate during video playback is calculated. The calculated frame rate is more reliable and accurate, avoiding statistical errors due to display limitations. This effectively solves the problem of incorrectly detecting frame rate and smoothness when the display refresh rate is limited or when data is not rendered in time, as is the existing method of evaluating smoothness by checking if the CPU and GPU time meets a predetermined frame rate threshold.

[0103] The following section uses the RGB values ​​of at least one test point as an example to illustrate the specific process of evaluating the playback performance of multimedia devices in Scenario 1 above.

[0104] Figure 5 A flowchart illustrating a playback performance evaluation method according to an embodiment of this disclosure is shown. Figure 5 As shown, the method may include the following steps:

[0105] Step 501: Send at least one RGB value corresponding to a test point to the multimedia device.

[0106] The number of test points can be configured according to requirements and is not limited.

[0107] For example, the evaluation device can iterate through all the preset test points and send the RGB values ​​corresponding to all test points to the multimedia device one by one.

[0108] For example, for different screen brightness levels of multimedia devices, the same set of RGB values ​​can be used to evaluate the display color accuracy and / or display brightness mapping effect of multimedia devices under different screen brightness levels; different sets of RGB values ​​can also be used to evaluate the display color accuracy and / or display brightness mapping effect of multimedia devices under different screen brightness levels, without limitation.

[0109] For example, the target display mode of a multimedia device can be obtained; and the RGB values ​​of at least one test point corresponding to the target display mode can be determined.

[0110] Step 502: Obtain the chromaticity and / or brightness of the multimedia device when it displays based on the RGB values ​​corresponding to the at least one test point.

[0111] For example, when the test point can be a node within the brightness range, the brightness presented by the multimedia device when displaying according to the RGB value corresponding to the test point can be obtained; when the test point can be a node within the color range, the chromaticity presented by the multimedia device when displaying the RGB value corresponding to the test point can be obtained.

[0112] For example, in step 502 above, the evaluation device can send the RGB values ​​corresponding to the test points to be tested to the multimedia device in real time. After the multimedia device successfully displays the content according to the RGB values ​​corresponding to the test points, it can send feedback information to the evaluation device so that the evaluation device can drive the color analyzer to start the acquisition work after the multimedia device displays the content according to the RGB values ​​corresponding to the test points, thereby acquiring the chromaticity and / or brightness presented by the multimedia device when displaying the content according to the values ​​corresponding to each test point.

[0113] Step 503: Evaluate the display color accuracy and / or display brightness mapping effect of the multimedia device based on the chromaticity and / or brightness presented when the multimedia device displays according to the RGB values ​​corresponding to the at least one test point.

[0114] Color accuracy refers to the precision of displayed colors. As an example, chromaticity can be encoded using the CIE LUV model. The LUV model, proposed by the International Commission on Illumination (ICI) in 1976, is derived from the CIE XYZ color space through transformation and possesses visual uniformity. In LUV, "L" represents pixel brightness, while "U" and "V" represent chromaticity. Furthermore, the chromaticity (uv) values ​​displayed by the multimedia device based on the RGB values ​​corresponding to each test point, collected by a color analyzer, are compared with the uv values ​​of the CIE 1931 color space standard to calculate the color deviation (ΔUV). This color deviation is then used to evaluate the color accuracy of the multimedia device. For example, when the color deviation exceeds a preset threshold, it indicates poor color accuracy of the multimedia device; conversely, it indicates good color accuracy.

[0115] For example, the color shift ΔUV can be calculated according to the following formula:

[0116] △UV=sqrt(pow(U-standard_U,2)+pow(V–standard_V,2))

[0117] In the above formula, U represents the chromaticity U value presented by the multimedia device when displaying the RGB values ​​corresponding to at least one test point, V represents the chromaticity V value presented by the multimedia device when displaying the RGB values ​​corresponding to at least one test point, standard_U represents the standard U value of the CIE1931 color space, and standard_V represents the standard chromaticity V value of the CIE1931 color space. Sqrt() represents the square root calculation function, and pow() represents the power function calculation.

[0118] The display brightness mapping effect represents the accuracy of the display brightness, reflecting the brightness performance capability of the multimedia device. As an example, firstly, the input pixel value of each test point on the multimedia device is obtained, and the brightness component can be calculated using the following formula:

[0119] Yi = Kr*Ri + Kg*Kg*Gi + Kb*Bi

[0120] In the above formula, Yi is the input brightness value of test point i (that is, the original brightness value of test point i), Ri is the R color value of test point i, Gi is the G color value of test point i, Bi is the B color value of test point i, Kr is the coefficient corresponding to the Ri color value, Kg is the coefficient corresponding to the Gi color value, and Kb is the coefficient corresponding to the Bi color value. For example, Kr = 0.229, Kg = 0.6917, and Kb = 0.0793.

[0121] Then, the output brightness value of each test point is read from the brightness displayed by the multimedia device based on the RGB values ​​corresponding to each test point, collected by the color analyzer. This process is repeated for all test points to obtain the input and output brightness values ​​of the multimedia device at each test point. Based on these values, the input-output brightness mapping curve corresponding to the multimedia device's output image can be determined. The slope and compression factor of this curve are analyzed to determine the contrast and performance of the multimedia device in low brightness, mid-gray, and high brightness ranges. This information reflects the brightness performance capability of the multimedia device. The compression factor refers to the ratio between the output range and the input range for each brightness interval; it can be categorized as low brightness, mid-gray, and high brightness. Low brightness refers to a lower brightness range, mid-gray refers to a medium brightness range, and high brightness refers to a higher brightness range.

[0122] Furthermore, the brightness mapping curve can be compared with the standard perceptual quantizer (PQ) curve to calculate the amount of brightness information loss corresponding to each test point. The brightness information loss value is then compared with the expected threshold to evaluate the display brightness mapping effect of the multimedia device. For example, when the brightness information loss value differs significantly from the expected threshold, it indicates that the display brightness mapping effect of the multimedia device is poor; conversely, it indicates that the display brightness mapping effect of the multimedia device is good.

[0123] In this embodiment of the disclosure, the display color accuracy and / or display brightness mapping effect of the multimedia device are evaluated based on the chromaticity and / or brightness presented by the multimedia device when displaying according to the RGB values ​​corresponding to at least one test point. As an example, the evaluation device pushes RGB values ​​to the multimedia device and controls the color analyzer to collect the chromaticity and brightness presented by the multimedia device when displaying according to the RGB values. The collected chromaticity is compared with the chromaticity of the CIE1931 color space, and the brightness mapping curve generated based on the collected brightness is compared with the PQ curve, thereby realizing the evaluation of the display color accuracy and brightness mapping effect of the multimedia device.

[0124] The following section uses the test data as the second video test sequence as an example to explain the specific process of evaluating the playback performance of the multimedia device in the above scenario one.

[0125] Figure 6 A flowchart illustrating a playback performance evaluation method according to an embodiment of this disclosure is shown. Figure 6 As shown, the method may include the following steps:

[0126] Step 601: Send the second video test sequence to the multimedia device.

[0127] In this second video test sequence, at least one frame contains a solid color region. The solid color region can be a relatively small area within a video frame; exemplarily, all frames in the second video test sequence contain solid color regions. Exemplarily, the color of the solid color regions in different video frames of the second video test sequence can all be white. For example, the R, G, and B values ​​of the solid color regions in the same video frame are equal to represent white, while the R, G, and B values ​​of the solid color regions in different video frames are different to represent white with different brightness levels.

[0128] Considering that HDR has a wider dynamic range of exposure than ordinary digital imaging technology, it can accurately represent the brightness range corresponding to direct sunlight to the darkest shadow in the real world. To more effectively evaluate the HDR performance of multimedia devices, the second video test sequence can, for example, include video sequences with obvious low-brightness, mid-gray, and high-brightness characteristics. For example, it can be a combination of low-brightness video frames, mid-grayness video frames, and high-brightness video frames arranged in a desired manner. For example, based on the characteristics of the video sequence, numerical values ​​representing the characteristics can be extracted, and several consecutive points around the edges can be identified as test points. For example, for a high-brightness characteristic brightness of 800, points 794-806 can be extracted as test points. In the second video test sequence, the solid-color areas of different frames in the high-brightness video frames sequentially display these test points; for example, displaying white indicating the brightness of the test point.

[0129] For example, a target display mode of a multimedia device can be obtained; and a second video test sequence corresponding to the target display mode can be determined.

[0130] Step 602: Obtain the chromaticity and / or brightness of the solid color area when the multimedia device plays the second video test sequence.

[0131] For example, the evaluation device can control the color analyzer to collect the brightness of the solid color area when the multimedia device plays the second video test sequence, thereby sequentially obtaining the brightness of each test point when the multimedia device displays the solid color area of ​​different frames when playing the second video test sequence.

[0132] Step 603: Evaluate the high dynamic range (HDR) performance of the multimedia device based on the chromaticity and / or brightness of the solid color area when the multimedia device plays the second video test sequence.

[0133] For example, the brightness of solid color areas when a multimedia device plays a video sequence with obvious low-brightness features can be used to determine whether there are cases where dark areas are not visible; the brightness of solid color areas when a multimedia device plays a video sequence with obvious mid-grayness features can be used to determine whether mid-grayness is too bright or too dark; and the brightness of solid color areas when a multimedia device plays a video sequence with obvious high-brightness features can be used to determine whether there are cases where bright areas are overexposed and unclear. In this way, by using video sequences with obvious low-brightness, mid-grayness, and high-brightness features, the video content is more diverse, thereby enabling a more comprehensive evaluation of the HDR performance of the multimedia device.

[0134] For example, the HDR performance of a multimedia device can be evaluated based on the brightness of the solid color area when the multimedia device plays the second video test sequence. For instance, the input and output brightness values ​​at each test point can be calculated, and then, based on these values, the input-output brightness mapping curve corresponding to the output image of the multimedia device can be determined. The slope and compression factor of this curve can be analyzed to determine the contrast ratio of the multimedia device for low brightness, mid-gray, and high brightness, thereby evaluating the HDR performance. The specific method for calculating the input and output brightness values ​​at each test point can be referred to above. Figure 5 The relevant statements in step 503 will not be repeated here.

[0135] In this embodiment, the high dynamic range (HDR) performance of a multimedia device is evaluated based on the chroma and / or brightness of a solid color region when the multimedia device plays a second video test sequence. As an example, considering that the device may modify the chroma and brightness according to different scenes during playback, a second video test sequence with distinct low-brightness, mid-gray, and high-brightness characteristics is provided, resulting in diverse and rich video content. Furthermore, at least one frame in the second video test sequence contains a solid color region. The chroma and brightness of this solid color region are then collected using a color analyzer, thereby accurately evaluating the HDR performance of the multimedia device. Compared to evaluating HDR performance using a complete solid color image, this embodiment allows for configuring a small area in the video frame to display a solid color in different video scenarios. When the multimedia device plays the video, the chroma and brightness of this small area are measured. Thus, even if the multimedia device uses different brightness mappings in different video scenarios, the HDR performance of the multimedia device can still be accurately evaluated using the chroma and brightness of this small area.

[0136] The following section uses the third video test sequence and / or audio test sequence as examples to illustrate the specific process of evaluating the playback performance of multimedia devices in the above scenario one.

[0137] Figure 7 A flowchart illustrating a playback performance evaluation method according to an embodiment of this disclosure is shown. Figure 7 As shown, the method may include the following steps:

[0138] Step 701: Send a third video test sequence and / or audio test sequence to the multimedia device.

[0139] For example, a target display mode of a multimedia device can be obtained; and a third video test sequence and / or audio test sequence corresponding to the target display mode can be determined.

[0140] Step 702: Obtain user feedback information regarding the playback effect when the multimedia device plays the third video test sequence and / or audio test sequence.

[0141] For example, the evaluation device can send a prompt message to the user. When the user watches the multimedia device playing the third video test sequence or listens to the multimedia device playing the audio test sequence, they can confirm whether the playback effect of the multimedia device is good based on the prompt message, and feed back the confirmation information to the evaluation device as the user's feedback information on the playback effect when the multimedia device plays the third video test sequence and / or audio test sequence. For example, the evaluation device can present the prompt message to the user through a UI interface and receive the user's confirmation information.

[0142] Step 703: Evaluate the playback performance of the multimedia device based on the user's feedback on the playback effect.

[0143] For example, the audio playback capability of a multimedia device can be evaluated based on user feedback regarding the audio playback effect when the multimedia device plays an audio test sequence. Similarly, the video playback capability of a multimedia device can be evaluated based on user feedback regarding the video playback effect when the multimedia device plays a third video test sequence.

[0144] In this embodiment of the disclosure, the playback performance of the multimedia device is evaluated based on the user's subjective feedback on the playback effect. As an example, objective data can be further combined to evaluate the playback performance of the multimedia device. In this way, by combining subjective feedback on the playback effect of the multimedia device with objective test data, the playback performance of the multimedia device is evaluated, thereby comprehensively assessing whether the playback performance of the multimedia device meets the expected level from both objective and subjective perspectives.

[0145] The following section uses test data as a preset brightness value as an example to explain the specific process of evaluating the playback performance of multimedia devices in the above scenario one.

[0146] Figure 8 A flowchart illustrating a playback performance evaluation method according to an embodiment of this disclosure is shown. Figure 8 As shown, the method may include the following steps:

[0147] Step 801: Send a preset brightness value to the multimedia device.

[0148] For example, a preset number of consecutive test points can be selected in the dark area and the bright area within the brightness range as the peak brightness and the minimum brightness, respectively. The preset number can be set according to the size of the brightness range and the requirements. For example, if the brightness range is 0 to 1023, 15 consecutive test points can be selected in the dark area of ​​0-300 as the minimum brightness, and 15 consecutive test points can be selected in the bright area of ​​700-1023 as the peak brightness.

[0149] For example, the target display mode of the multimedia device can be obtained; and a preset brightness value corresponding to the target display mode can be determined.

[0150] Step 802: Obtain the brightness displayed when the multimedia device displays the specific brightness value.

[0151] For example, the evaluation device can acquire the brightness of the multimedia device when it displays peak brightness and the brightness of the multimedia device when it displays minimum brightness. For instance, it can acquire the brightness collected by a color analyzer when the multimedia device displays a preset number of consecutive test points in the aforementioned dark and bright areas.

[0152] Step 803: Evaluate the quantification accuracy of the display brightness and the smoothness of brightness changes of the multimedia device based on the brightness presented when the multimedia device displays the specific brightness value.

[0153] For example, the dispersion coefficients (such as variance or mean) of the brightness gradient curve can be calculated based on the brightness presented when the multimedia device displays the specific brightness value; then it can be determined whether the gradient curve rises smoothly or in a step-like manner, thereby assessing the smoothness of the brightness change.

[0154] For example, the dispersion coefficient of the brightness gradient curve can be compared with a preset threshold to evaluate whether the quantization accuracy of the display brightness of the multimedia device is qualified. If the dispersion coefficient is less than the preset threshold, the quantization accuracy is deemed qualified; if the dispersion coefficient is greater than the preset threshold, the quantization accuracy is deemed unqualified. The specific value of the preset threshold can be set according to requirements; for example, the preset threshold can be 1.1.

[0155] In this embodiment of the disclosure, the quantification accuracy of the display brightness of the multimedia device and the smoothness of brightness changes are evaluated by using a preset brightness value, thereby enabling a more comprehensive evaluation of the playback performance of the multimedia device.

[0156] Figure 9 A flowchart illustrating a playback performance evaluation method according to an embodiment of the present disclosure is shown. This method can be derived from the above... Figure 1 and Figure 2 The evaluation system shown is executed as follows: Figure 9 As shown, the method may include the following steps:

[0157] Step 901: Multimedia device 20 starts the test application.

[0158] For example, a test application can be launched in response to a user's operation on the multimedia device 20. This test application may be pre-installed on the multimedia device 20. For example, a user may download and install the test application onto the multimedia device 20 from the evaluation device 10 or service center 30 via a network.

[0159] Step 902: Evaluate the establishment of a communication network connection between the evaluation device 10 and the multimedia device 20.

[0160] For example, the multimedia device 20 starts the network communication module and starts the multimedia device 20 test application to evaluate the communication information transmitted by the device 10 to the multimedia device 20, establish a network connection, and obtain the current display mode, basic information (chip, width and height, refresh rate, decoder information, etc.) of the multimedia device 20, and collect the post-processing capabilities of the multimedia device 20.

[0161] Step 903: Initialize the color analyzer 40 for evaluation equipment 10.

[0162] For example, the color analyzer 40 is aligned with the multimedia device 20 so that the color analyzer 40 can collect the chromaticity and brightness of the multimedia device 20 in real time, and the evaluation device 10 can simultaneously acquire the chromaticity and brightness collected by the color analyzer 40 in real time.

[0163] Step 904: Evaluation device 10 sends out RGB values.

[0164] For example, the evaluation device 10 traverses the test data and sends the RGB values ​​corresponding to the test points to the multimedia device 20 one by one.

[0165] Step 905: Multimedia device 20 displays a success message.

[0166] For example, the multimedia device 20 displays the corresponding color and grayscale on its screen based on the received RGB values ​​and notifies the evaluation device 10 that the display has been completed.

[0167] Step 906: Evaluate device 10 to obtain chromaticity and luminance.

[0168] For example, the evaluation device 10 drives the color analyzer 40 to read and record the chromaticity and brightness displayed by the multimedia device 20.

[0169] Step 907: Color analyzer 40 reports chromaticity and luminance.

[0170] For example, the above steps 604-606 are executed repeatedly until the color analyzer 40 collects the chromaticity and luminance corresponding to all test points displayed by the multimedia device 20 and reports them to the evaluation device 10; then the color accuracy and luminance mapping effect of the multimedia device 20 are evaluated.

[0171] Step 908: Evaluation device 10 pushes video test sequence to multimedia device 20.

[0172] For example, the evaluation device 10 pushes a first video test sequence to the multimedia device 20 and plays the first video test sequence on the multimedia device 20; the color analyzer 40 reads the brightness value waveform of the multimedia device 20.

[0173] For example, the evaluation device 10 pushes peak brightness, minimum brightness, UI brightness, quantization accuracy, and multiple HDR test video sequences to the multimedia device 20 and plays them on the multimedia device 20, while the color analyzer 40 collects the corresponding chromaticity and brightness.

[0174] For example, the evaluation device 10 pushes a second video test sequence, such as an SDR or HDR test sequence, to the multimedia device 20.

[0175] For example, the evaluation device 10 pushes a third video test sequence and / or audio test sequence to the multimedia device 20, which includes video sequences with different decoders, bit depths, resolutions, and frame rates, and different audio formats.

[0176] Step 909: Evaluate the data acquired by the equipment 10.

[0177] For example, the evaluation device 10 requests to obtain a waveform of the brightness value; it also requests to obtain the chromaticity and brightness corresponding to the peak brightness, minimum brightness, and UI brightness displayed by the multimedia device 20.

[0178] For example, the evaluation device 10 evaluates the chromaticity and luminance presented when it requests the multimedia device 20 to play a second video test sequence.

[0179] For example, the evaluation device 10 evaluates the chromaticity and luminance presented when it requests the multimedia device 20 to play a third video test sequence.

[0180] Step 910: Color analyzer 40 reports data.

[0181] For example, the color analyzer 40 reports a waveform of the brightness value; reports the chromaticity and brightness corresponding to the peak brightness, minimum brightness, and UI brightness displayed by the multimedia device 20; reports the chromaticity and brightness presented when the multimedia device 20 plays the second video test sequence; and reports the chromaticity and brightness presented when the multimedia device 20 plays the third video test sequence.

[0182] Step 911, Evaluate the device 10, evaluate smoothness, HDR performance, device support, etc.

[0183] For example, the evaluation device 10 analyzes the brightness value waveform, obtains the frame rate statistics by calculating the Fourier transform, calculates the number of red and green wavelengths, accurately reads the actual rendering frame rate, and determines the smoothness of the video played by the multimedia device 20.

[0184] For example, the evaluation device 10 calculates the brightness information loss based on the chromaticity and brightness corresponding to the peak brightness, minimum brightness, and UI brightness displayed by the storage multimedia device 20.

[0185] For example, the evaluation device 10 evaluates whether the HDR performance of the multimedia device 20 meets expectations based on the chroma and brightness presented when the multimedia device 20 plays the second video test sequence.

[0186] Step 912: Evaluate device 10 to obtain user feedback information.

[0187] For example, the multimedia device 20 sequentially plays a second video test sequence and / or an audio test sequence. During the playback, the user can fill in feedback information based on their subjective viewing experience.

[0188] Step 913: Evaluate equipment 10 to generate data pending approval.

[0189] For example, the evaluation device 10 controls the multimedia device 20 to switch to another display mode, and after switching to the new display mode, repeats the above steps 902-912 to test the new display mode; thereby generating data to be approved.

[0190] Step 914: Evaluate the data reported by device 10.

[0191] For example, the evaluation device 10 summarizes the information tested in each display mode and submits it to the service center 300.

[0192] Step 915, Service Center 30 Trial Data.

[0193] For example, the service center 30 determines the multimedia playback performance of the multimedia device 20 based on the information from tests conducted under each display mode and the evaluation results from the evaluation device 10; it can also generate targeted adjustment suggestions and conclusions, as well as generate reports (such as certification reports).

[0194] Step 916: Evaluate equipment 10 to obtain conclusions and reports.

[0195] For example, the evaluation device 10 obtains the conclusions and reports generated by the service center 30, completes the test, and can further optimize the playback performance of the multimedia device 20 based on the conclusions and reports.

[0196] In this embodiment, the evaluation supports performance assessment of playback under different display modes of a multimedia device. A communication connection is established between the evaluation device and the multimedia device. For each display mode, the evaluation device controls the multimedia device to play test data and acquires the output information of the multimedia device when playing the test data. This allows for a comprehensive evaluation of the multimedia device's basic playback capabilities, decoding compatibility, actual rendering capabilities, audio playback capabilities, color accuracy, brightness mapping effects, HDR performance, and post-processing capabilities through rich output information.

[0197] Based on the same inventive concept as the above method embodiments, embodiments of this disclosure also provide a playback performance evaluation device, which can be used to execute the technical solutions described in the above method embodiments. For example, it can execute the above... Figures 3-9 The steps of the playback performance evaluation method shown are as follows.

[0198] Figure 10 This diagram illustrates a structural diagram of a playback performance evaluation apparatus according to an embodiment of the present disclosure, applied to an evaluation device, such as... Figure 10 As shown, the device includes: a sending module 1001, used to send test data to the multimedia device, wherein the evaluation device establishes a communication connection with the multimedia device; and an acquisition module 1002, used to acquire output information corresponding to the multimedia device; the output information includes information presented by the multimedia device when playing according to the test data; wherein the output information is used to evaluate the playback performance of the multimedia device.

[0199] In this embodiment, a communication connection is established between the evaluation device and the multimedia device. The evaluation device controls the multimedia device to play data based on test data and acquires the output information of the multimedia device during playback. The output information is used to evaluate the multimedia playback performance of the multimedia device, and the evaluation result reflects the true playback performance of the multimedia device. As an example, the evaluation device can use the output information to comprehensively evaluate the multimedia device's basic playback capabilities, decoding compatibility, actual rendering capabilities, audio playback capabilities, color accuracy, brightness mapping effects, HDR performance, post-processing capabilities, etc.

[0200] In one possible implementation, the test data includes a first video test sequence; the output information includes information presented when the multimedia device plays the first video test sequence; wherein the information presented when the multimedia device plays the first video test sequence is used to evaluate the smoothness of the video playback by the multimedia device.

[0201] In one possible implementation, the first video test sequence includes multiple video frames corresponding to solid colors, wherein video frames corresponding to different solid colors are arranged alternately in the first video test sequence.

[0202] In one possible implementation, the information presented by the multimedia device when playing the first video test sequence includes a luminance value waveform; wherein, evaluating the smoothness of the video playback by the multimedia device includes: obtaining frame rate statistics by performing a Fourier transform on the luminance value waveform; the frame rate statistics represent the frequency of luminance value changes in the luminance value waveform; determining the number of frames corresponding to each pure color wavelength among the plurality of pure colors based on the frame rate statistics; and determining the rendering frame rate of the multimedia device according to the number of frames corresponding to each pure color wavelength, so as to evaluate the smoothness of the video playback by the multimedia device.

[0203] In one possible implementation, the test data includes RGB values ​​corresponding to at least one test point; the output information includes the chromaticity and / or brightness presented by the multimedia device when displaying based on the RGB values ​​corresponding to the at least one test point; wherein the chromaticity and / or brightness presented by the multimedia device when displaying based on the RGB values ​​corresponding to the at least one test point is used to evaluate the display color accuracy and / or display brightness mapping effect of the multimedia device.

[0204] In one possible implementation, the test data includes a second video test sequence; at least one frame in the second video test sequence contains a solid color region; the output information includes: the chroma and / or brightness of the solid color region when the multimedia device plays the second video test sequence; wherein the chroma and / or brightness of the solid color region when the multimedia device plays the second video test sequence is used to evaluate the high dynamic range (HDR) performance of the multimedia device.

[0205] In one possible implementation, the test data includes a third video test sequence and / or an audio test sequence; the output information includes user feedback on the playback effect when the multimedia device plays the third video test sequence and / or audio test sequence; wherein the user feedback on the playback effect is used to evaluate the playback performance of the multimedia device.

[0206] In one possible implementation, the sending module 1001 is further configured to: send a collection command to the color analyzer to instruct the color analyzer to collect information presented by the multimedia device when playing according to the test data; the acquisition module is further configured to: receive the information presented by the multimedia device when playing according to the test data collected by the color analyzer.

[0207] In one possible implementation, the sending module 1001 is further configured to: send a first instruction, a preset color value, and the test data to the multimedia device at a first moment, wherein the first instruction is used to instruct the multimedia device to display the preset color and prepare to play according to the test data; the acquiring module 1002 is further configured to: acquire the original presentation information of the multimedia device at fixed intervals, and send a second instruction to the multimedia device at a second moment, wherein the second instruction is used to instruct the multimedia device to start playing according to the test data; and, based on the original presentation information and the preset color, filter out valid messages from the original presentation information as the corresponding output information of the multimedia device.

[0208] In one possible implementation, the acquisition module 1002 is further configured to: acquire the target display mode of the multimedia device; the test data corresponds to the target display mode.

[0209] The above Figure 10 The technical effects and specific descriptions of the playback performance evaluation device and its various possible implementations can be found in the above-mentioned playback performance evaluation method, and will not be repeated here.

[0210] It should be understood that the division of modules in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the modules in the device can be implemented by a processor calling software; for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each module in the device. The processor can be, for example, a general-purpose processor, such as a Central Processing Unit (CPU) or a microprocessor, and the memory can be internal or external to the device. Alternatively, the modules in the device can be implemented as hardware circuits. The functionality of some or all modules can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC). The functionality of some or all of the modules is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it can include a large number of logic gates. The connection relationships between these logic gates are configured through configuration files, thereby achieving the functionality of some or all of the modules. All modules of the above device can be implemented entirely through processor-invoked software, entirely through hardware circuits, or partially through processor-invoked software with the remaining parts implemented through hardware circuits.

[0211] In this disclosure, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a CPU, microprocessor, graphics processing unit (GPU), digital signal processor (DSP), neural network processing unit (NPU), tensor processing unit (TPU), etc. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules.

[0212] As can be seen, each module in the above apparatus can be one or more processors (or processing circuits) configured to implement the methods of the above embodiments, such as: CPU, GPU, NPU, TPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types. Furthermore, each module in the above apparatus can be integrated in whole or in part, or can be implemented independently; there is no limitation on this.

[0213] This disclosure also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of the above embodiments when executing the instructions. Exemplarily, the above can be performed. Figures 3-9 The steps of the playback performance evaluation method are shown below.

[0214] Figure 11 This diagram illustrates the structure of an electronic device according to an embodiment of the present disclosure, such as... Figure 11 As shown, the electronic device may include at least one processor 1501, a communication line 1502, a memory 1503, and at least one communication interface 1504.

[0215] Processor 1501 may be a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits for controlling the execution of programs according to the present disclosure; processor 1501 may also include a heterogeneous computing architecture of multiple general-purpose processors, for example, it may be a combination of at least two of CPU, GPU, microprocessor, DSP, ASIC, and FPGA; as an example, processor 1501 may be CPU+GPU or CPU+ASIC or CPU+FPGA.

[0216] Communication line 1502 may include a path for transmitting information between the aforementioned components.

[0217] Communication interface 1504 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

[0218] The memory 1503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via communication line 1502. The memory may also be integrated with the processor. The memory provided in this embodiment of the present disclosure is generally non-volatile. The memory 1503 is used to store computer execution instructions for executing the scheme of the present disclosure and is controlled by the processor 1501 for execution. The processor 1501 is used to execute computer execution instructions stored in the memory 1503, thereby implementing the method provided in the above embodiments of this disclosure; exemplarily, the above can be implemented. Figures 3-9 The steps of the playback performance evaluation method shown are as follows.

[0219] Optionally, the computer execution instructions in this embodiment may also be referred to as application code, and this embodiment does not specifically limit this.

[0220] For example, processor 1501 may include one or more CPUs, e.g. Figure 11 CPU0 in the CPU; processor 1501 may also include a CPU, and any one of GPU, ASIC, or FPGA, for example, Figure 11 The CPU0+GPU0, CPU0+ASIC0, or CPU0+FPGA0 are mentioned.

[0221] For example, an electronic device may include multiple processors, such as Figure 11 Processors 1501 and 1507 are mentioned. Each of these processors can be a single-core processor, a multi-core processor, or a heterogeneous computing architecture that includes multiple general-purpose processors. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0222] In a specific implementation, as one embodiment, the electronic device may further include an output device 1505 and an input device 1506. The output device 1505 communicates with the processor 1501 and can display information in various ways. For example, the output device 1505 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, such as a vehicle-mounted HUD, AR-HUD, or monitor. The input device 1506 communicates with the processor 1501 and can receive user input in various ways. For example, the input device 1506 may be a mouse, keyboard, touchscreen device, or sensing device.

[0223] Embodiments of this disclosure provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the methods described in the above embodiments. Exemplarily, the above... Figures 3-9 The steps of the playback performance evaluation method shown are as follows.

[0224] Embodiments of this disclosure provide a computer program product, which may include, for example, computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code; when the computer program product is run on a computer, the computer performs the methods described in the above embodiments. Exemplarily, the above... Figures 3-9 The steps of the playback performance evaluation method shown are as follows.

[0225] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0226] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0227] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0228] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0229] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0230] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0231] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0232] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0233] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for evaluating playback performance, characterized in that, Applied to an evaluation device, the method includes: Test data is sent to a multimedia device, wherein a communication connection is established between the evaluation device and the multimedia device; Obtain the output information corresponding to the multimedia device; the output information includes the information presented by the multimedia device when playing the test data. The output information is used to evaluate the playback performance of the multimedia device; The evaluation of the playback performance of the multimedia device includes: obtaining frame rate statistics by performing a Fourier transform on the brightness value waveform of the multimedia device when playing a first video test sequence; the frame rate statistics represent the frequency of brightness value changes in the brightness value waveform; the first video test sequence includes multiple video frames corresponding to solid colors. Based on the frame rate statistics, the number of frames corresponding to each wavelength of the multiple solid colors is determined; The rendering frame rate of the multimedia device is determined based on the number of frames corresponding to each pure color wavelength, in order to evaluate the smoothness of video playback on the multimedia device.

2. The method according to claim 1, characterized in that, The test data includes a first video test sequence; the output information includes information presented when the multimedia device plays the first video test sequence. The information presented when the multimedia device plays the first video test sequence is used to evaluate the smoothness of the video playback on the multimedia device.

3. The method according to claim 2, characterized in that, The first video test sequence includes multiple video frames corresponding to solid colors, wherein video frames corresponding to different solid colors are arranged alternately in the first video test sequence.

4. The method according to any one of claims 1-3, characterized in that, The test data includes the RGB values ​​of at least one test point; the output information includes the chromaticity and / or brightness presented by the multimedia device when displaying based on the RGB values ​​of the at least one test point. The chromaticity and / or brightness presented by the multimedia device when displaying based on the RGB values ​​corresponding to the at least one test point are used to evaluate the display color accuracy and / or display brightness mapping effect of the multimedia device.

5. The method according to any one of claims 1-3, characterized in that, The test data includes a second video test sequence; at least one frame in the second video test sequence contains a solid color region; the output information includes: the chromaticity and / or brightness of the solid color region when the multimedia device plays the second video test sequence; The chromaticity and / or brightness of the solid color region when the multimedia device plays the second video test sequence are used to evaluate the high dynamic range (HDR) performance of the multimedia device.

6. The method according to any one of claims 1-3, characterized in that, The test data includes a third video test sequence and / or an audio test sequence; the output information includes user feedback on the playback effect when the multimedia device plays the third video test sequence and / or audio test sequence; The user feedback on the playback effect is used to evaluate the playback performance of the multimedia device.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: Send a collection command to the color analyzer to instruct the color analyzer to collect the information presented by the multimedia device when playing the test data; The step of obtaining the output information corresponding to the multimedia device includes: The information collected by the color analyzer when the multimedia device plays the test data is received.

8. The method according to any one of claims 1-3, characterized in that, Sending test data to the multimedia device includes: At a first moment, a first instruction, a preset color value, and the test data are sent to the multimedia device. The first instruction is used to instruct the multimedia device to display the preset color and prepare to play according to the test data. The step of obtaining the output information corresponding to the multimedia device includes: The original presentation information of the multimedia device is acquired at fixed intervals, and a second instruction is sent to the multimedia device at a second moment. The second instruction is used to instruct the multimedia device to start playing according to the test data. Based on the original presentation information and the preset color, valid messages are selected from the original presentation information as the output information corresponding to the multimedia device.

9. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain the target display mode of the multimedia device; the test data corresponds to the target display mode.

10. A playback performance evaluation system, characterized in that, The system includes: evaluation equipment and multimedia equipment; The evaluation device is used to send test data to the multimedia device; and a communication connection is established between the evaluation device and the multimedia device. The multimedia device is used to receive the test data and play it according to the test data. The evaluation device is also used to acquire output information corresponding to the multimedia device; the output information includes information presented when the multimedia device plays according to the test data. The output information is used to evaluate the playback performance of the multimedia device. The evaluation of the playback performance includes: obtaining frame rate statistics by performing a Fourier transform on the brightness value waveform of the multimedia device playing a first video test sequence; the frame rate statistics represent the frequency of brightness value changes in the brightness value waveform; the first video test sequence includes multiple video frames corresponding to solid colors; based on the frame rate statistics, determining the number of frames corresponding to each solid color wavelength among the multiple solid colors; and determining the rendering frame rate of the multimedia device according to the number of frames corresponding to each solid color wavelength, in order to evaluate the smoothness of video playback on the multimedia device.

11. The system according to claim 10, characterized in that, The system also includes: a service center; The service center is used to update the test data and synchronize the updated test data to the evaluation device; The evaluation device is also used to evaluate the playback performance of the multimedia device based on the output information, and report the output information and the evaluation results of the playback performance of the multimedia device to the service center; The service center is also used to acquire the output information and the results of evaluating the playback performance of the multimedia device; and based on the output information and the results of evaluating the playback performance of the multimedia device, generate a multimedia playback performance report of the multimedia device, and feed the multimedia playback performance report back to the evaluation device. The evaluation device is also used to receive the multimedia playback performance report.

12. The system according to claim 10, characterized in that, The system also includes: a color analyzer; The evaluation device is also used to send acquisition commands to the color analyzer; The color analyzer is used to respond to the acquisition command, acquire information presented by the multimedia device when playing according to the test data, and feed the acquired information back to the evaluation device.

13. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 9 when executing instructions stored in the memory.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 9.

Images