Coding and decoding frame rate visualization method and device, electronic equipment and storage medium

Abstract

The invention provides a coding and decoding frame rate visualization method and device, electronic equipment and a storage medium, and relates to the technical field of electronic equipment. Comprising the following steps: acquiring a reference frame rate corresponding to audio and video data being played in the electronic equipment; determining a target frequency according to the reference frame rate; based on the target frequency, reading a first frame rate when the codec encodes and decodes the audio and video data; generating a frame rate curve graph according to the first frame rate and first time when the first frame rate is read; and displaying the frame rate curve graph in a display screen. Therefore, the coding and decoding frame rates of the codec can be visualized through the frame rate curve graph, and the performance of the codec is determined according to the frame rate curve graph, so that the coding and decoding frame rates of the codec can be monitored intuitively in real time.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic devices, and in particular, to a method, apparatus, electronic device, and storage medium for visualizing codec frame rates. Background Art

[0002] With the rapid development of the Internet and digital technologies, the transmission and processing of media content such as audio, video, and images have become increasingly common and important. Codecs are software or hardware tools used to convert raw media data into a compressed format. They are widely used in various applications, such as video conferencing, streaming media, broadcast television, gaming, and mobile communications.

[0003] Therefore, how to display the codec frame rate in real time during the codec encoding and decoding process has become an important research direction. Summary of the Invention

[0004] The present disclosure aims to at least solve one of the technical problems in the related art to some extent.

[0005] A first aspect embodiment of the present disclosure provides a method for visualizing codec frame rates, including:

[0006] Obtaining a reference frame rate corresponding to the audio-visual data being played on the electronic device;

[0007] Determining a target frequency according to the reference frame rate;

[0008] Based on the target frequency, reading a first frame rate when the codec encodes and decodes the audio-visual data;

[0009] Generating a frame rate curve graph according to the first frame rate and the first time when the first frame rate is read;

[0010] Displaying the frame rate curve graph on the display screen.

[0011] A second aspect embodiment of the present disclosure provides a device for visualizing codec frame rates, including:

[0012] An obtaining module, configured to obtain a reference frame rate corresponding to the audio-visual data being played on the electronic device;

[0013] A first determining module, configured to determine a target frequency according to the reference frame rate;

[0014] A reading module, configured to read a first frame rate when the codec encodes and decodes the audio-visual data based on the target frequency;

[0015] A generating module, configured to generate a frame rate curve graph according to the first frame rate and the first time when the first frame rate is read;

[0016] A display module, configured to display the frame rate curve graph on a display screen.

[0017] A third aspect embodiment of the present disclosure provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the visualization method for the encoding / decoding frame rate as proposed in the first aspect embodiment of the present disclosure.

[0018] A fourth aspect embodiment of the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the visualization method for the encoding / decoding frame rate as proposed in the first aspect embodiment of the present disclosure.

[0019] The visualization method, device, electronic device, and storage medium for the encoding / decoding frame rate provided by the present disclosure have the following beneficial effects:

[0020] In the embodiments of the present disclosure, first, the reference frame rate corresponding to the audio-visual data being played on the electronic device is obtained. Then, based on the reference frame rate, the target frequency is determined. Based on the target frequency, the first frame rate when the codec encodes and decodes the audio-visual data is read. Further, according to the first frame rate and the first time when the first frame rate is read, a frame rate curve graph is generated. Finally, the frame rate curve graph is displayed on the display screen. Thus, the encoding / decoding frame rate of the codec can be visualized through the frame rate curve graph, so that the encoding / decoding frame rate of the codec can be monitored intuitively and in real time.

[0021] The additional aspects and advantages of the present disclosure will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present disclosure. Description of the Drawings

[0022] The above and / or additional aspects and advantages of the present disclosure will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0023] Figure 1 is a schematic flowchart of a visualization method for an encoding / decoding frame rate provided by an embodiment of the present disclosure;

[0024] Figure 2 is a frame rate curve graph provided by an embodiment of the present disclosure;

[0025] Figure 3 is a schematic flowchart of a visualization method for an encoding / decoding frame rate provided by an embodiment of the present disclosure;

[0026] Figure 4 is a frame rate curve graph provided by another embodiment of the present disclosure;

[0027] Figure 5A frame rate curve diagram provided in another embodiment of the present disclosure;

[0028] Figure 6 An interaction schematic diagram of a method for visualizing the encoding / decoding frame rate provided in an embodiment of the present disclosure

[0029] Figure 7 A structural schematic diagram of a device for visualizing the encoding / decoding frame rate provided in an embodiment of the present disclosure;

[0030] Figure 8 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present disclosure is shown. Detailed implementation manners

[0031] Embodiments of the present disclosure will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, and should not be construed as a limitation of the present disclosure.

[0032] The method, device, electronic device, and storage medium for visualizing the encoding / decoding frame rate according to the embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0033] In the embodiments of the present disclosure, the method for visualizing the encoding / decoding frame rate is exemplified by being configured in a device for visualizing the encoding / decoding frame rate. The device for visualizing the encoding / decoding frame rate can be applied to any electronic device so that the electronic device can perform the function of visualizing the encoding / decoding frame rate.

[0034] In some embodiments, the method for visualizing the encoding / decoding frame rate in the embodiments of the present disclosure can be executed by an electronic device (such as a TV, computer, mobile phone, etc.) including an encoder / decoder itself. Alternatively, it can also be executed by other electronic devices. For example, other electronic devices can be connected to the electronic device including the encoder / decoder in a wired manner to achieve the visualization of the encoding / decoding frame rate.

[0035] Figure 1 A flowchart of a method for visualizing the encoding / decoding frame rate provided in an embodiment of the present disclosure.

[0036] As Figure 1 shown, the method for visualizing the encoding / decoding frame rate may include the following steps:

[0037] Step 101, obtain a reference frame rate corresponding to the audio-visual data being played in the electronic device.

[0038] Among them, the audio-visual data may be audio data, video data, video and video data, etc., and the present disclosure does not make a limitation thereto.

[0039] Among them, the reference frame rate can be configured by an audio and video playback application in an electronic device for the played audio and video data, such as 30 frames per second (fps), 60 fps, etc. The present disclosure does not limit this.

[0040] In some embodiments, when the visualization method of the codec frame rate is executed by other electronic devices outside the electronic device (for example, the host computer software VideoChartPC), the other electronic device can obtain the reference frame rate from the visualization program (VideoChartSYS) of the electronic device. Among them, the reference frame rate in the visualization program is sent by the audio and video application that plays the audio and video data to the visualization program.

[0041] Specifically, VideoChartPC first creates a command-line interface (CLI) thread, and then can set the property named video.debug.enable-frameRate-debug to 1 through the Android Debug Bridge, which means to turn it on, and send it to the visualization program VideoChartSYS in the electronic device containing the codec, notifying it to enable the frame rate visualization function, that is, to output the video codec frame rate debug information. This property only needs to be set once.

[0042] After that, when VideoChartSYS creates MediaCodec in the video playback application, that is, when the video playback application requests to play a video, it can read the value of video.debug.enable-frameRate-debug. When the value read is 1, it enables the calculation of the video codec frame rate. After the video playback application configures the reference frame rate, VideoChartSYS can read the configured reference frame rate from the audio and video playback application (Video APP). Finally, it can send the reference frame rate to VideoChartPC by setting properties, or VideoChartPC can read the reference frame rate from VideoChartSYS by itself.

[0043] Step 102, determine the target frequency according to the reference frame rate.

[0044] Optionally, the reciprocal of the reference frame rate can be determined as the target frequency.

[0045] It should be noted that if the video playback application device sets the reference frame rate (frame-rate) corresponding to the audio and video data, then the time interval for the codec to output the codec frame rate is at least 1 / frame-rate seconds. Therefore, the reciprocal of the reference frame rate is determined as the target frequency, so that the codec frame rate of the codec can be accurately captured.

[0046] Alternatively, the target frame rate can also be slightly greater than the reciprocal of the reference frame rate, so as to save resources while ensuring that not too many codec frame rates are missed. However, if the target frequency is set too large, it may cause VideoChartPC not to capture all the codec frame rates.

[0047] Step 103: Based on the target frequency, read the first frame rate when the codec encodes and decodes the audio and video data.

[0048] The codec can be a device in the electronic device for encoding and decoding audio and video data. For example, an audio codec or a video codec.

[0049] The first frame rate can be the codec's encoding and decoding frame rate.

[0050] In some embodiments, after obtaining the reference frame rate (frame-rate), VideoChartPC can create a timer thread. Every 1 / frame-rate seconds, it can obtain the first frame rate by setting the property debug.video.decoder-fps.

[0051] In some embodiments, VideoChartSYS can calculate the first frame rate by calculating the rendering time interval between two frames of audio and video data.

[0052] Optionally, the first frame rate is determined by the visualization program in the electronic device based on the second time when the current frame of audio and video data is encoded and decoded and the third time when the previous frame of audio and video data is encoded and decoded.

[0053] The calculation formula for the first frame rate can be:

[0054]

[0055] Where Vf represents the first frame rate of video encoding and decoding, with the unit of fps. Tcodec represents the second time when the current frame of audio and video data is encoded and decoded, with the unit of μs. Tcodec’ is the third time when the previous frame of audio and video data is encoded and decoded, with the unit of μs.

[0056] Step 104: Generate a frame rate curve graph based on the first frame rate and the first time when the first frame rate is read.

[0057] The first time can be the system time in the device for monitoring the codec performance.

[0058] Optionally, a frame rate curve graph can be generated with the first time when the first frame rate is read as the abscissa and the first frame rate as the ordinate.

[0059] In some embodiments, the frame rate curve graph may be generated only based on the first frame rate obtained within a preset time period before the current moment. For example, the frame rate within 5 seconds, the frame rate within 3 seconds, etc. may be displayed through the frame rate curve graph, and the present disclosure does not limit this.

[0060] Step 105, display the frame rate curve graph on the display screen.

[0061] In the embodiments of the present disclosure, after the frame rate curve graph is generated, the frame rate curve graph may be displayed on the display, so as to realize the visualization of the codec frame rate, enabling the user to view the codec frame rate of the codec in real time through the frame rate curve graph, that is, the actual frame rate of the audio and video data being played.

[0062] In some embodiments, if the frame rate curve graph is generated by an electronic device (such as a TV, a computer, a mobile phone, etc.) including a codec, the frame rate curve graph may be displayed on the display screen of the electronic device including the codec.

[0063] In some embodiments, if the frame rate curve graph is generated by another electronic device connected to the electronic device including the codec through a wire, the frame rate curve graph may be displayed on the display screen of the other electronic device connected to the electronic device including the codec through a wire.

[0064] Figure 2 A frame rate curve graph provided by an embodiment of the present disclosure is shown as Figure 2 shown, the abscissa is time, the ordinate is the frame rate, the reference frame rate is 30fps, and the real-time frame rate curve fluctuates above and below the reference frame rate.

[0065] In the embodiments of the present disclosure, first, the reference frame rate corresponding to the audio and video data being played in the electronic device is obtained, then the target frequency is determined according to the reference frame rate, and based on the target frequency, the first frame rate when the codec encodes and decodes the audio and video data is read; furthermore, the frame rate curve graph is generated according to the first frame rate and the first time when the first frame rate is read, and finally the frame rate curve graph is displayed on the display screen. Thus, the codec frame rate of the codec can be visualized through the frame rate curve graph, so that the codec frame rate of the codec can be monitored intuitively and in real time.

[0066] Figure 3 A schematic flowchart of a method for visualizing the codec frame rate provided by an embodiment of the present disclosure is shown as Figure 3 shown, and the method for visualizing the codec frame rate may include the following steps:

[0067] Step 301, obtain the reference frame rate corresponding to the audio and video data being played in the electronic device.

[0068] Step 302, determine the target frequency according to the reference frame rate.

[0069] Step 303: Based on the target frequency, read the first frame rate when the codec encodes and decodes the audio and video data.

[0070] For the specific implementation forms of steps 301 to 303, reference can be made to the detailed steps in other embodiments of the present disclosure, which will not be elaborated herein.

[0071] Step 304: Obtain a preset number of the first frame rates read before the current moment.

[0072] Optionally, the preset number can be determined according to the maximum working frame rate of the codec.

[0073] Among them, the maximum working frame rate can be the upper limit of the encoding and decoding frame rate of the codec.

[0074] Specifically, the maximum working frame rate can be determined as the preset number. For example, if the maximum working frame rate is 80fps, the preset number can be 80.

[0075] Alternatively, the preset number can also be determined according to the reference frame rate. The larger the reference frame rate, the larger the preset number can be.

[0076] Alternatively, the preset number can also be a pre-configured value. Specifically, the maximum encoding and decoding frame rate currently available on the market can be determined as the preset number. For example, if the maximum encoding and decoding frame rate is 120fps, the preset number can be set to 120. Thus, at most 120 first frame rates can be read within one second, and 120 first frame rates are displayed in the frame rate curve graph, which is more helpful for determining the performance of the codec.

[0077] Step 305: Draw a frame rate curve graph based on the preset number of the first frame rates and the first time when each first frame rate is read.

[0078] Specifically, using the first time corresponding to each first frame rate as the abscissa and the first frame rate as the ordinate, determine the positions corresponding to the preset number of the first frame rates, and then generate a frame rate curve to obtain a frame rate curve graph.

[0079] In some embodiments, as time changes, the curve is continuously updated. Each time a first frame rate is obtained, a frame rate point is defined with the system time when the first frame rate is read as the abscissa and the first frame rate value as the ordinate, and then the frame rate point is inserted into the curve. After the number of frame rate points reaches the preset number, the graph is refreshed. The refreshing method can be to delete the first point and then insert the newly added point into the curve.

[0080] In some embodiments, after the first frame rate is read, the first frame rate and the first time corresponding to the first frame rate can also be stored offline, so that after the audio and video data is played, the offline data can also be analyzed.

[0081] Step 306: Display a frame rate curve graph on the display screen.

[0082] Step 307: Determine the target fault level of the codec according to the frame rate curve graph and the reference frame rate.

[0083] In some embodiments, as Figure 2 shown, if the frame rate curve in the frame rate curve graph fluctuates above and below the reference frame rate, it is determined that the performance of the codec is normal.

[0084] In some embodiments, if the frame rate curve in the frame rate curve graph does not fluctuate above and below the reference frame rate, it is determined that the codec may have a fault, such as a performance degradation or non - operation, etc.

[0085] Optionally, in response to the reference frame rate being greater than the maximum peak value of the frame rate curve in the frame rate curve graph and the difference between the reference frame rate and the maximum peak value being greater than the first threshold, determine the first fault level of the target fault level.

[0086] Wherein, the first threshold can be 3fps, 2fps, etc.

[0087] Wherein, the first fault level can be a performance degradation of the decoder.

[0088] It can be understood that if the maximum peak value is still lower than the reference frame rate and the difference is greater than the first threshold, it means that the encoding and decoding frame rate of the codec does not reach the expected reference frame rate and the performance does not meet the standard.

[0089] Figure 4 A frame rate curve graph provided by another embodiment of the present disclosure. As Figure 4 shown, the reference frame rate is 30fps, and each first frame rate in the frame rate curve is lower than the reference frame rate, indicating that the encoding and decoding performance does not meet the standard. The actual performance may be that the audio - video playback application plays smoothly.

[0090] Alternatively, in response to there being a line segment in the frame rate curve where the frame rate remains unchanged and the corresponding duration is greater than the second threshold, and the absolute value of the difference between the frame rate corresponding to the line segment and the reference frequency is less than the third threshold, determine that the target fault level is the second fault level.

[0091] Wherein, the second threshold can be 0.5 seconds, 1 second, 3 seconds, etc. The third threshold can be 3fps, 5fps, etc.

[0092] Wherein, the second fault level can be that the codec is broken and cannot perform encoding and decoding.

[0093] It should be noted that when the absolute value of the difference between the frame rate corresponding to the line segment and the reference frequency is less than the third threshold, it can ensure that the frame rate corresponding to the line segment is near the reference frame rate, so that the reason for the appearance of a straight line can be determined as a codec failure. If the absolute value of the difference is greater than the third threshold, there may still be a problem of performance degradation.

[0094] Figure 5 A frame rate curve graph provided by another embodiment of the present disclosure. As Figure 5 shown, the dashed box represents a straight line, indicating that the codec may not be working, resulting in the frame rate value not being refreshed, thereby determining that the codec has failed.

[0095] In the embodiment of the present disclosure, first, the reference frame rate corresponding to the audio and video data being played in the electronic device is obtained. Then, based on the reference frame rate, the target frequency is determined. Based on the target frequency, the first frame rate when the codec encodes and decodes the audio and video data is read. Furthermore, a preset number of first frame rates read before the current moment are obtained. Based on the preset number of first frame rates and the first time when each first frame rate is read, a frame rate curve graph is drawn and the frame rate curve graph is displayed. Finally, based on the frame rate curve graph and the reference frame rate, the target failure level of the codec is determined. Thus, a preset number of first frame rates can be displayed in the frame rate curve graph, and then the target failure level of the codec can be determined according to the frame rate curve graph, thereby avoiding too many or too few first frame rates in the frame rate curve graph, resulting in an inaccurate determination of the target failure level of the codec, and improving the accuracy of target failure level monitoring.

[0096] Figure 6 An interaction schematic diagram of a method for visualizing codec frame rates provided by an embodiment of the present disclosure. As Figure 6 shown, the method for visualizing codec frame rates includes:

[0097] Step 601, VideoChartPC instructs VideoChartSYS to enable the frame rate visualization function.

[0098] Step 602, Video APP creates a MediaCodec object and sends the reference frame rate to VideoChartSYS.

[0099] Step 603, VideoChartPC reads the reference frame rate from VideoChartSYS.

[0100] Step 604, VideoChartPC determines the target frequency according to the reference frame rate.

[0101] Step 605, VideoChartSYS determines the first frame rate according to the second time when the current frame of audio and video data is encoded and decoded and the third time when the previous frame of audio and video data is encoded and decoded.

[0102] Step 606, the VideoChartPC reads the first frame rate when the codec encodes and decodes the audio and video data based on the target frequency.

[0103] Specifically, by setting the attribute debug.video.decoder-fps, the first frame rate is read from the obtained VideoChartPC.

[0104] Step 607, the VideoChartPC generates a frame rate curve graph according to the first frame rate and the first time when the first frame rate is read.

[0105] Step 608, the VideoChartPC displays the frame rate curve graph on the display screen.

[0106] Step 609, the Video APP sends a stop encoding and decoding instruction to the VideoChartSYS.

[0107] Step 610, the VideoChartSYS sends a stop signal to the VideoChartPC.

[0108] In some embodiments, after receiving the stop encoding and decoding instruction, the VideoChartSYS no longer calculates the first frame rate and sets video.debug.enable-frameRate-debug to 0.

[0109] Step 611, the VideoChartPC stops generating the frame rate curve graph.

[0110] To implement the above embodiments, the present disclosure also proposes a visualization device for the encoding and decoding frame rate.

[0111] Figure 7 It is a schematic structural diagram of the visualization device for the encoding and decoding frame rate provided by the embodiments of the present disclosure.

[0112] As Figure 7 shown, the visualization device 700 for the encoding and decoding frame rate may include:

[0113] An acquisition module 701, configured to acquire a reference frame rate corresponding to the audio and video data being played in the electronic device;

[0114] A first determination module 702, configured to determine a target frequency according to the reference frame rate;

[0115] A reading module 703, configured to read the first frame rate when the codec encodes and decodes the audio and video data based on the target frequency;

[0116] A generation module 704, configured to generate a frame rate curve graph according to a first frame rate and a first time when the first frame rate is read;

[0117] A display module 705, configured to display the frame rate curve graph on a display screen.

[0118] In some embodiments, a generation module 604 is configured to:

[0119] Obtain a preset number of first frame rates read before the current moment;

[0120] Draw a frame rate curve based on the preset number of first frame rates and the first time when each first frame rate is read.

[0121] In some embodiments, it further includes a second determination module, configured to:

[0122] Determine the preset number according to the maximum working frame rate of the codec; or,

[0123] Determine the preset number according to a reference frame rate.

[0124] In some embodiments, a first determination module 602 is configured to:

[0125] Determine the reciprocal of the reference frame rate as the target frequency.

[0126] In some embodiments, the first frame rate is determined by a visualization program in an electronic device according to a second time when current frame audio-visual data is completed by codec and a third time when previous frame audio-visual data is completed by codec.

[0127] In some embodiments, an acquisition module 701 is configured to:

[0128] Obtain a reference frame rate from a visualization program in the electronic device, where the reference frame rate in the visualization program is sent by an audio-visual application that plays the audio-visual data to the visualization program.

[0129] In some embodiments, it further includes a third determination module, configured to:

[0130] Determine a target fault level of the codec according to the frame rate curve graph and the reference frame rate.

[0131] In some embodiments, the third determination module is configured to:

[0132] In response to the reference frame rate being greater than the maximum peak value of the frame rate curve in the frame rate curve graph and the difference between the reference frame rate and the maximum peak value being greater than a first threshold, determine a first fault level of the target fault level; or,

[0133] In response to the presence of a line segment with a constant frame rate, a corresponding duration greater than a second threshold, and an absolute value of the difference between the frame rate corresponding to the line segment and a reference frequency less than a third threshold in the frame rate curve, it is determined that the target fault level is the second fault level, where the fault degree of the second fault level is higher than that of the first fault level.

[0134] For the functions and specific implementation principles of the above-mentioned modules in the embodiments of the present disclosure, reference may be made to the above-mentioned method embodiments, which will not be elaborated here.

[0135] The codec frame rate visualization device according to the embodiments of the present disclosure first obtains a reference frame rate corresponding to the audio-video data being played in an electronic device, then determines a target frequency according to the reference frame rate, and based on the target frequency, reads the first frame rate when the codec encodes and decodes the audio-video data; furthermore, according to the first frame rate and the first time when the first frame rate is read, a frame rate curve graph is generated, and finally the frame rate curve graph is displayed on the display screen. Thus, the codec frame rate can be visualized through the frame rate curve graph, so that the codec frame rate can be monitored intuitively and in real time.

[0136] To implement the above embodiments, the present disclosure also proposes an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the codec frame rate visualization method proposed in the foregoing embodiments of the present disclosure.

[0137] To implement the above embodiments, the present disclosure also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the codec frame rate visualization method proposed in the foregoing embodiments of the present disclosure.

[0138] Figure 8 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present disclosure is shown. Figure 8 The illustrated electronic device 12 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0139] As Figure 8 shown, the electronic device 12 is presented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0140] Bus 18 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, an Accelerated Graphics Port, a processor bus, or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnection (PCI) bus.

[0141] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including both volatile and nonvolatile media, removable and non-removable media.

[0142] Memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / nonvolatile computer system storage media. By way of example only, storage system 34 can be used for reading and writing on non-removable, nonvolatile magnetic media ( Figure 8 not shown, typically called a "hard disk drive"). Although Figure 8 not shown in, a disk drive for reading and writing on a removable nonvolatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing on a removable nonvolatile optical disk (such as Compact Disc Read Only Memory (CD-ROM), Digital Video Disc Read Only Memory (DVD-ROM), or other optical media) can be provided. In these instances, each drive can be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of the various embodiments of the present disclosure.

[0143] A program / utilities 40 having a set (at least one) of program modules 42 can be stored, for example, in a memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules 42 generally execute the functions and / or methods in the embodiments described in this disclosure.

[0144] The electronic device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 22. Moreover, the electronic device 12 can also communicate with one or more networks (such as a Local Area Network (LAN), a Wide Area Network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0145] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the foregoing embodiments.

[0146] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0147] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0148] Any process or method description represented in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logical function or process. The scope of the preferred embodiments of the present disclosure includes additional implementations, where functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed. This should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0149] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with such instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0150] It should be understood that various parts of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logic functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0151] Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0152] In addition, in each of the embodiments of the present disclosure, the functional units can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the above integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0153] The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disc, or the like. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for visualizing the encoding / decoding frame rate, characterized in that, it includes: Obtain the reference frame rate corresponding to the audio-visual data being played in the electronic device; Determine the target frequency according to the reference frame rate; Based on the target frequency, read the first frame rate when the codec encodes and decodes the audio-visual data; Generate a frame rate curve graph according to the first frame rate and the first time when the first frame rate is read; Display the frame rate curve graph on the display screen.

2. The method according to claim 1, characterized in that, The generating a frame rate curve graph according to the first frame rate and the first time when the first frame rate is read includes: Obtain a preset number of first frame rates read before the current moment; Draw the frame rate curve graph based on the preset number of first frame rates and the first time when each first frame rate is read.

3. The method according to claim 2, characterized in that, It further includes: Determine the preset number according to the maximum working frame rate of the codec; or, Determine the preset number according to the reference frame rate.

4. The method according to claim 1, characterized in that, The determining the target frequency according to the reference frame rate includes: Determine the reciprocal of the reference frame rate as the target frequency.

5. The method according to claim 1, characterized in that, The first frame rate is determined by the visualization program in the electronic device according to the second time when the current frame of audio-visual data is encoded and decoded and the third time when the previous frame of audio-visual data is encoded and decoded.

6. The method according to claim 1, characterized in that, The obtaining the reference frame rate corresponding to the audio-visual data being played in the electronic device includes: Obtain the reference frame rate from the visualization program of the electronic device, wherein the reference frame rate in the visualization program is sent by the audio-visual application that plays the audio-visual data to the visualization program.

7. The method according to claim 1, characterized in that, After displaying the frame rate curve graph on the display screen, it further includes: Determine the target fault level of the codec according to the frame rate curve graph and the reference frame rate.

8. The method according to claim 7, characterized in that, The determining the target fault level of the codec according to the frame rate curve graph and the reference frame rate includes: In response to the reference frame rate being greater than the maximum peak value of the frame rate curve in the frame rate curve graph and the difference between the reference frame rate and the maximum peak value being greater than the first threshold, determine the first fault level of the target fault level; or, In response to there being a line segment in the frame rate curve where the frame rate remains unchanged and the corresponding duration is greater than the second threshold, and the absolute value of the difference between the frame rate corresponding to the line segment and the reference frequency is less than the third threshold, determine that the target fault level is the second fault level, wherein the fault degree of the second fault level is higher than the fault degree of the first fault level.

9. A visualization device for encoding / decoding frame rate, characterized in that, The device includes: An obtaining module, configured to obtain the reference frame rate corresponding to the audio-visual data being played in the electronic device; A first determination module, configured to determine a target frequency according to the reference frame rate; A reading module, configured to read a first frame rate when the codec encodes and decodes the audio and video data based on the target frequency; A generating module, configured to generate a frame rate curve graph according to the first frame rate and a first time when the first frame rate is read; A display module, configured to display the frame rate curve graph on a display screen.

10. The apparatus according to claim 9, wherein, the generating module is configured to: obtain a preset number of first frame rates read before the current moment; draw the frame rate curve graph based on the preset number of first frame rates and a first time when each of the first frame rates is read.

11. The apparatus according to claim 10, wherein, it further includes a second determination module, configured to: determine the preset number according to a maximum working frame rate of the codec; or determine the preset number according to the reference frame rate.

12. The apparatus according to claim 9, wherein, the first determination module is configured to: determine the reciprocal of the reference frame rate as the target frequency.

13. The apparatus according to claim 9, wherein, the first frame rate is determined by a visualization program in the electronic device according to a second time when the current frame of audio and video data is encoded and decoded, and a third time when the previous frame of audio and video data is encoded and decoded.

14. The apparatus according to claim 9, wherein, the obtaining module is configured to: obtain the reference frame rate from a visualization program of the electronic device, where the reference frame rate in the visualization program is sent by an audio and video application that plays the audio and video data to the visualization program.

15. The apparatus according to claim 9, wherein, it further includes a third determination module, configured to: determine a target fault level of the codec according to the frame rate curve graph and the reference frame rate.

16. The apparatus according to claim 15, wherein, the third determination module is configured to: in response to the reference frame rate being greater than a maximum peak value of a frame rate curve in the frame rate curve graph and a difference between the reference frame rate and the maximum peak value being greater than a first threshold, determine a first fault level of the target fault level; or in response to there being a line segment in the frame rate curve where the frame rate remains unchanged and a corresponding duration is greater than a second threshold, and an absolute value of a difference between a frame rate corresponding to the line segment and the reference frequency is less than a third threshold, determine that the target fault level is a second fault level, where a fault degree of the second fault level is higher than a fault degree of the first fault level.

17. An electronic device, wherein, it includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the visualization method for the codec frame rate as described in any one of claims 1-8.

18. A computer-readable storage medium, storing a computer program, wherein, When the computer program is executed by a processor, it implements the visualization method of the encoding / decoding frame rate as described in any one of claims 1-8.

Images