Image data processing method and device, electronic equipment and medium

Abstract

The invention discloses an image data processing method and device, electronic equipment and a medium, and belongs to the technical field of data processing. The method comprises the steps of outputting image data according to a first frame rate under the condition that a frame insertion function is started and a touch event is detected, and caching the image data; under the condition that the frame insertion function is started and a touch event is not detected, frame pressing processing is executed, image data of a second frame rate after frame pressing processing is obtained, the image data after frame pressing processing is cached, frame insertion processing is conducted on the image data after frame pressing processing, and the image data after frame insertion processing is output, the frame rate of the image data after frame insertion processing is the first frame rate, and the second frame rate is smaller than the first frame rate.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to an image data processing method, apparatus, electronic device, and medium. Background Technology

[0002] With the rapid development of mobile communication and electronic technologies, electronic devices have become indispensable communication tools in people's daily lives, and are also frequently used for entertainment activities such as games and videos. In gaming and video usage scenarios on electronic devices, existing technologies often employ frame interpolation to reduce power consumption. For example, the game frame rate is first compressed to a base frame rate, such as 60fps, and then frame interpolation is performed to restore it to 120fps. However, this process has a problem: the game engine sends a frame rate of 60fps after compression (frame interval of 16.67ms), while the user ultimately perceives a 120fps frame generated by frame interpolation (frame interval of 8.33ms), resulting in a frame rate discrepancy. This discrepancy causes significant latency for users during touch operations. Although the smoothness of the screen is improved, the operation response speed is still limited by the 60fps input frame rate, leading to a decrease in responsiveness. Therefore, existing frame interpolation technology causes a disconnect between operation and display latency, resulting in a certain degree of operational feedback delay. Summary of the Invention

[0003] The purpose of this application is to provide an image data processing method, apparatus, electronic device, and medium that can solve the problem of certain operation feedback delay when the existing frame interpolation function is enabled.

[0004] In a first aspect, embodiments of this application provide an image data processing method, the method comprising: When frame interpolation is enabled and a touch event is detected, image data is output at the first frame rate and the image data is cached. When frame interpolation is enabled and no touch event is detected, frame compression is performed to obtain image data with a second frame rate after frame compression. The image data after frame compression is cached, frame interpolation is performed on the image data after frame compression, and the image data after frame interpolation is output. The frame rate of the image data after frame interpolation is the first frame rate, and the second frame rate is less than the first frame rate.

[0005] Secondly, embodiments of this application provide an image data processing apparatus, including: The first processing module is used to output image data according to the first frame rate and cache the image data when the frame interpolation function is enabled and a touch event is detected; The second processing module is used to perform frame compression processing when the frame interpolation function is enabled and no touch event is detected, to obtain image data with a second frame rate after frame compression processing, to cache the image data after frame compression processing, to perform frame interpolation processing on the image data after frame compression processing, and to output the image data after frame interpolation processing, wherein the frame rate of the image data after frame interpolation processing is the first frame rate, and the second frame rate is less than the first frame rate.

[0006] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0009] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0010] In this embodiment, when frame interpolation is enabled and a touch event is detected, image data is output at a first frame rate and the image data is cached. When frame interpolation is enabled but no touch event is detected, frame compression is performed to obtain image data at a second frame rate after frame compression, and the compressed image data is cached. Frame interpolation is then performed on the compressed image data, and the interpolated image data is output. The frame rate of the interpolated image data is the first frame rate, and the second frame rate is less than the first frame rate. Thus, for frame interpolation scenarios, frame compression is only performed when no touch event is detected. When a touch event is detected, frame compression can be omitted to output image data at a higher frame rate, thereby reducing or avoiding touch operation feedback delay and significantly improving operation responsiveness. Attached Figure Description

[0011] Figure 1 This is a flowchart of the image data processing method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the frame interpolation processing strategy provided in the embodiments of this application; Figure 3a and Figure 3b This is a flowchart of two threads in the frame compression control process provided in the embodiments of this application; Figure 4a This is a flowchart of the frame compression thread provided in an embodiment of this application; Figure 4b This is a flowchart of the frame interpolation algorithm thread provided in the embodiments of this application; Figure 5 This is a structural diagram of the image data processing apparatus provided in the embodiments of this application; Figure 6 This is a structural diagram of the electronic device provided in the embodiments of this application; Figure 7 This is a hardware structure diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] The image data processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0015] Please see Figure 1 , Figure 1 A flowchart of the image data processing method provided in the embodiments of this application is shown below. Figure 1 As shown, the method includes the following steps: Step 101: With frame interpolation enabled and a touch event detected, output image data at the first frame rate and cache the image data.

[0016] The embodiments of this application can be applied to scenarios where electronic devices enable frame interpolation, such as when playing games or watching videos on electronic devices. In these scenarios, electronic devices will use frame interpolation technology to compress the image data to be output before interpolating the frames to reduce power consumption.

[0017] The aforementioned touch event can refer to an event detected on the touchscreen of an electronic device where a user's finger touches the screen to perform a related operation, that is, the user's touch operation is detected.

[0018] In this embodiment, when the frame interpolation function is enabled, it can monitor for touch events. If a touch event is detected, frame compression can be omitted, and image data can be output at the original first frame rate, i.e., using a higher frame rate to output image data to the graphics display system, such as using a high frame rate of 120 FPS. Furthermore, the currently output image data can be added to a buffer queue for use when frame interpolation is needed in the next frame.

[0019] In some embodiments, the first frame rate can be the frame rate without frame compression, that is, the original frame rate, such as 120 FPS. In this way, it can be ensured that the frame rate sent to the display and the frame rate perceived by the user are both the original frame rate, so there will be no frame rate disconnection. There will be no delay when the user performs touch operations, and the operation response speed will not be limited.

[0020] In some embodiments, the first frame rate can also be a frame rate that has undergone some frame compression, but is higher than the second frame rate after frame compression. For example, if the original frame rate is 120 FPS, the second frame rate is 60 FPS, and the first frame rate is 100 FPS, then the original frame rate can be slightly reduced to obtain a higher first frame rate, and then frames can be added later to increase it to the original frame rate for display. This can also reduce the touch operation feedback latency to some extent.

[0021] It should be noted that here, the output image data can be directly output to the graphics display system at the original frame rate, or the image data can be sent to the frame interpolation module first. When the frame interpolation module determines that the first frame rate is the original frame rate, it chooses not to perform frame interpolation processing and directly outputs the image data to the graphics display system. Alternatively, when the frame interpolation module determines that the first frame rate is not the original frame rate, it restores the original frame rate by inserting a small number of frames before outputting it to the graphics display system.

[0022] Optionally, when the frame interpolation function is enabled and a touch event is detected, outputting image data according to the first frame rate includes: When a touch event is detected under the condition that frame interpolation is enabled and the user is in the first usage scenario, image data is output according to the first frame rate. The first usage scenario includes at least one of the following: game scenario and video playback scenario.

[0023] In some embodiments, the embodiments of this application can be enabled only in specific usage scenarios, such as gaming or video playback scenarios, where touch event monitoring is activated only. When a touch event is detected in these scenarios, image data is output according to the first frame rate. This avoids unnecessary resource overhead caused by enabling touch event monitoring in some non-essential usage scenarios, and also ensures reduced touch operation latency in scenarios with high requirements for touch operation latency.

[0024] Step 102: With frame interpolation enabled and no touch event detected, perform frame compression processing to obtain image data at the second frame rate after frame compression processing, and cache the image data after frame compression processing. Perform frame interpolation processing on the image data after frame compression processing, and output the image data after frame interpolation processing. The frame rate of the image data after frame interpolation processing is the first frame rate, and the second frame rate is less than the first frame rate. The second frame rate is the frame rate obtained after compressing the original frame rate output by the application.

[0025] In this embodiment of the application, when the frame interpolation function is enabled and no touch event is detected, the frame compression process is performed normally. For example, the original high frame rate is compressed by half to obtain image data with a second frame rate, that is, the image data is output using a lower frame rate. The second frame rate can be half of the original frame rate. For example, if the original frame rate is 120 FPS before frame compression, the second frame rate is 60 FPS.

[0026] It should be noted that the image data after frame compression can be sent to the frame interpolation module, and the frame image data can be added to the buffer queue. Then, the frame interpolation module performs frame interpolation on the image data after frame compression before sending it for display, so as to restore the image data to the original frame rate before frame compression, such as the first frame rate. The frame interpolation module then outputs the frame-interpolated image data to the graphics display system for display. In this way, the image data can be displayed at a higher refresh rate on the front end, improving the smoothness of the picture.

[0027] Optionally, the method further includes: Upon detecting a touch event, the first time is set to the touch time corresponding to the touch event; If no touch event is detected, the first time is set to the first value; When frame interpolation is enabled and a touch event is detected, outputting image data according to the first frame rate includes: When frame interpolation is enabled, the first time is not equal to the first value, and the difference between the second time and the first time is less than the preset frame compression duration, image data is output according to the first frame rate. The step of performing frame compression processing when frame interpolation is enabled and no touch event is detected includes: When frame interpolation is enabled, the first time is equal to the first value, or the difference between the second time and the first time is greater than or equal to the preset frame compression duration, frame compression processing is performed. The second time is the refresh time of the image data.

[0028] In some embodiments, the touch event can be determined by recording the occurrence time of the touch event and the refresh time of the image data.

[0029] Specifically, the initial value of the first time T1 can be set to a first value. When no touch event is detected, the first time T1 is the first value, which can be less than or equal to 0. When a touch event is detected, the first time T1 is updated to the time when the touch event occurred, that is, the corresponding touch time. At this time, T1 is not the first value, and the touch time is greater than 0.

[0030] When no touch event occurs, the difference between the image data refresh time T2 (that is, the time it takes for the system to output image data to the buffer queue each time) and the first time T1 is greater than the preset frame compression duration △T. Therefore, frame compression processing can be performed by determining that there is no touch event when the first time T1 is equal to the first value, or when the difference between the second time T1 and the first time T1 is greater than or equal to the preset frame compression duration △T.

[0031] When a touch event occurs, the first time T1 is set as the time when the touch event occurs. At this time, the first time T1 is not equal to the first value, and the difference between the time T2 when the system outputs image data each time and the first time T1 is less than the preset frame compression duration. Therefore, it can be determined that there is a touch event when the first time T1 is not equal to the first value and the difference between the second time T1 and the first time T1 is less than the preset frame compression duration ΔT. At this time, frame compression processing can be omitted, and image data can be output according to the first frame rate.

[0032] In this way, touch events can be accurately detected by recording the time of occurrence, and then touch operation latency can be reduced by frame compression processing control.

[0033] In some embodiments, the preset frame compression duration is determined based on the frame compression rate, and the first value is 0.

[0034] That is, the preset frame compression duration △T can be set according to the frame compression rate. For example, if the frame rate after frame compression is 60FPS, the corresponding frame interval is 16.67ms. The preset frame compression duration △T can be set to 16ms, and the initial value of the first time T1, i.e. the first value, can be 0.

[0035] Thus, when there is no touch event, since T1 is 0, the difference between the time T2 and T1 for each time the system outputs image data must be greater than 16ms. Therefore, T1!=0 && T2-T1< △T is judged as failing. At this time, the system compresses the frame and outputs the image at a frame rate of 60FPS.

[0036] When a touch event occurs, set T1 to the time when the touch event occurs. At this time, T1 != 0 && T2-T1 < △T. The judgment is passed. At this time, the system does not compress the frame and outputs the image at a frame rate of 120 FPS.

[0037] Optionally, the step of performing frame interpolation on the compressed image data and outputting the interpolated image data includes: Retrieve the cached image data of the previous frame; Based on the previous frame image data and the compressed image data, calculate the intermediate frame image data, and replace the cached previous frame image data with the compressed image data; The intermediate frame image data and the compressed frame image data are output to the graphics display system in sequence.

[0038] In some embodiments, the image data output by the system each time can be cached in a buffer queue, such as a bufferqueue. When frame interpolation is needed, the frame interpolation module can retrieve the previous frame image data from the buffer and calculate the intermediate frame image data based on the retrieved previous frame image data and the current frame's compressed image data. Simultaneously, the cached previous frame image data needs to be replaced with the current frame's compressed image data. Then, the calculated intermediate frame image data and the current frame's compressed image data are output sequentially to the graphics display system for display.

[0039] The frame interpolation module's processing strategy after receiving touch event points and image data is as follows: Figure 2 As shown, when there is a touch event, frame compression is not performed, and the received current frame image data is directly output and buffered. When there is no touch event, frame compression is performed first, and then frame interpolation is performed. The intermediate frame between the two frames is calculated using the image data of the previous frame in the buffer and the received image data. At the same time, the buffer is replaced with the received image data, and then the calculated intermediate frame image data and the received image data are output one after the other.

[0040] In this way, by inserting an intermediate frame between two adjacent frames using a frame interpolation algorithm, the image frame rate can be restored to the frame rate before frame compression.

[0041] In summary, a specific embodiment of this application may include the following steps: Step 301: After enabling the 60-frame interpolation 120-frame function, the specific frame compression processing logic is as follows: Figure 3a and Figure 3b As shown, the frame compression duration is set to △T = 16ms. When there is no touch, since T1 is 0, the difference between the time T2 and T1 when the system outputs image data each time, i.e., T2-T1, must be greater than 16ms. T1!=0 && T2-T1< △T, the judgment fails. At this time, the system compresses the frame and outputs the image at a frame rate of 60FPS.

[0042] Step 302: When a touch event occurs, set T1 to the time when the touch event occurs. At this time, T1!=0 && T2-T1<△T is considered passed. At this time, the system does not compress the frame and outputs the image at a frame rate of 120FPS.

[0043] Step 303: The frame interpolation algorithm adjusts its strategy after receiving image data and touch events, as follows: Figure 2 As shown, if a touch event exists, the image data is directly output and cached. If no touch event exists, frame interpolation is performed. The intermediate frame between the two frames is calculated using the image data of the previous frame in the cache and the received image data. At the same time, the cached data is replaced with the received image data. Then, the calculated intermediate image data and the received image data are output sequentially.

[0044] The purpose of this application is to provide a latency optimization scheme for frame interpolation scenarios, enabling zero latency increment in frame interpolation scenarios. This method is applicable to any frame interpolation scheme, including double or multiple frame interpolation, as well as different frame interpolation scenarios, including but not limited to games, videos, and different frame interpolation algorithms, such as discrete graphics card frame interpolation, graphics processing unit (GPU) frame interpolation, etc. This scheme can significantly optimize frame interpolation latency and improve user smoothness.

[0045] This application embodiment can dynamically change the APP's frame compression by judging the touch event point in the frame interpolation scenario, thereby reducing the latency introduced by the APP's frame compression; combined with the strategy adjustment of the frame interpolation algorithm, skipping frame interpolation can reduce the latency introduced by the frame interpolation algorithm; ultimately, both together achieve the goal of zero-latency incremental frame interpolation. Taking a game scenario as an example, the processes of the frame compression thread and the frame interpolation algorithm thread are as follows: Figure 4a and Figure 4b As shown, the main process is as follows: When the game enters the 60-frame-per-120 frame rate mode, if there are no touch events, the system compresses the frame rate to 60 frames. Because the frame interpolation algorithm is effective, the output is 120 frames after interpolation. Therefore, the image data is passed to the frame interpolation algorithm at the same time as the frame is compressed. When a user touches the screen, a touch event is generated. After detecting the touch event, the system adjusts the frame compression thread to not compress frames, allowing the image consuming the touch event to be output at a frame rate of 120 FPS. At the same time, the touch event information is passed to the frame interpolation algorithm along with the image data. After the frame interpolation algorithm receives the information sent by the system, if the information sent by the system contains touch event information, the image data is output immediately to ensure that the image data is displayed as quickly as possible; if the information sent by the system does not contain touch events, the intermediate frame image is calculated by frame interpolation, and then the intermediate frame image and the received image data are output sequentially.

[0046] This application provides a zero-latency optimization scheme adaptable to all frame interpolation algorithms. The method offers system frame rate control and frame interpolation algorithm strategies, which work together. During touch events, system frame rate control is applied to quickly consume touch events, ensuring no increased latency during frame interpolation and saving power during non-touch events. In both cases, 120 frames per second (fps) are output, improving smoothness and avoiding the latency introduced by frame interpolation. This scheme significantly optimizes latency by 3-4 synchronization signals (vsync) (approximately 20-30ms), achieving latency identical to the native 120fps when interpolating from 60fps, thus achieving zero latency increment during frame interpolation. Specifically, it offers the following three major technical effects: 1) Significantly improved latency: By outputting the current frame at key nodes, the touch operation response cycle is aligned with the native frame rate. Actual tests show that touch latency is reduced by approximately 3-4 vsync cycles (25ms at 120fps), resulting in a significant improvement in responsiveness.

[0047] 2) Zero-latency frame interpolation guarantee: This solution allows the latency after frame interpolation to be aligned with the original latency, and ultimately achieves 120 frames per second output without affecting the smoothness of the picture, thus supporting the zero-latency selling point.

[0048] 3) Compatibility advantages: No need to modify the game application layer logic, only timing control is implemented in the system display layer (SurfaceFlinger) and frame interpolation algorithm, adapting to the existing frame interpolation hardware architecture.

[0049] The image data processing method in this embodiment outputs image data at a first frame rate when a touch event is detected; and performs frame compression processing when no touch event is detected, outputting image data at a second frame rate after frame compression processing; wherein the second frame rate is less than the first frame rate. In this way, frame compression processing is only performed when no touch event is detected, and when a touch event is detected, frame compression can be omitted to output image data at a higher frame rate, thereby reducing or avoiding touch operation feedback delay and significantly improving operation responsiveness.

[0050] The image data processing method provided in this application can be executed by an image data processing device. This application uses an image data processing device executing the image data processing method as an example to illustrate the image data processing device provided in this application.

[0051] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of the image data processing apparatus provided in the embodiments of this application, as shown below. Figure 5 As shown, the image data processing apparatus 500 includes: The first processing module 501 is used to output image data according to the first frame rate and cache the image data when the frame interpolation function is enabled and a touch event is detected; The second processing module 502 is used to perform frame compression processing when the frame interpolation function is enabled and no touch event is detected, to obtain image data with a second frame rate after frame compression processing, to cache the image data after frame compression processing, to perform frame interpolation processing on the image data after frame compression processing, and to output the image data after frame interpolation processing, wherein the frame rate of the image data after frame interpolation processing is the first frame rate, and the second frame rate is less than the first frame rate.

[0052] Optionally, the image data processing apparatus 500 further includes: The setting module is used to set the first time to the touch time corresponding to the touch event when a touch event is detected; and to set the first time to a first value when no touch event is detected. The first processing module 501 is used to output image data according to the first frame rate when the frame interpolation function is enabled, the first time is not equal to the first value, and the difference between the second time and the first time is less than the preset frame compression duration. The second processing module 502 is used to perform frame compression processing when the frame interpolation function is enabled, the first time is equal to the first value, or the difference between the second time and the first time is greater than or equal to the preset frame compression duration. The second time is the refresh time of the image data.

[0053] Optionally, the preset frame compression duration is determined based on the frame compression rate, and the first value is 0.

[0054] Optionally, the second processing module 502 includes: The acquisition unit is used to acquire the cached image data of the previous frame; The calculation unit is used to calculate the intermediate frame image data based on the previous frame image data and the compressed frame image data, and replace the cached previous frame image data with the compressed frame image data. The output unit is used to output the intermediate frame image data and the compressed frame image data to the graphics display system sequentially.

[0055] Optionally, the first processing module 501 is used to detect a touch event when the frame interpolation function is enabled and the user is in a first usage scenario, and output image data according to a first frame rate. The first usage scenario includes at least one of the following: a game scenario and a video playback scenario.

[0056] Optionally, the first frame rate is the frame rate without compression.

[0057] The image data processing apparatus 500 in this embodiment outputs image data at a first frame rate and caches the image data when frame interpolation is enabled and a touch event is detected. When frame interpolation is enabled but no touch event is detected, it performs frame compression processing to obtain image data at a second frame rate after frame compression processing, caches the image data after frame compression processing, performs frame interpolation processing on the image data after frame compression processing, and outputs the image data after frame interpolation processing. The frame rate of the image data after frame interpolation processing is the first frame rate, and the second frame rate is less than the first frame rate. Thus, for frame interpolation scenarios, frame compression processing is only performed when no touch event is detected. When a touch event is detected, frame compression processing is not performed, allowing for the output of image data at a higher frame rate. This reduces or avoids touch operation feedback delay and significantly improves operation responsiveness.

[0058] The image data processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0059] The image data processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0060] The image data processing device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiment shown in Figure 4 can achieve the same technical effect, and will not be described again here to avoid repetition.

[0061] Optionally, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described image data processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0062] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0063] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0064] The electronic device 700 includes, but is not limited to, components such as: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

[0065] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0066] The processor 710 is used for: When frame interpolation is enabled and a touch event is detected, image data is output at the first frame rate and the image data is cached. When frame interpolation is enabled and no touch event is detected, frame compression is performed to obtain image data with a second frame rate after frame compression. The image data after frame compression is cached, frame interpolation is performed on the image data after frame compression, and the image data after frame interpolation is output. The frame rate of the image data after frame interpolation is the first frame rate, and the second frame rate is less than the first frame rate.

[0067] Optionally, the processor 710 is also used for: Upon detecting a touch event, the first time is set to the touch time corresponding to the touch event; If no touch event is detected, the first time is set to the first value; When frame interpolation is enabled, the first time is not equal to the first value, and the difference between the second time and the first time is less than the preset frame compression duration, image data is output according to the first frame rate. When frame interpolation is enabled, the first time is equal to the first value, or the difference between the second time and the first time is greater than or equal to the preset frame compression duration, frame compression processing is performed. The second time is the refresh time of the image data.

[0068] Optionally, the preset frame compression duration is determined based on the frame compression rate, and the first value is 0.

[0069] Optionally, the processor 710 is also used for: Retrieve the cached image data of the previous frame; Based on the previous frame image data and the compressed image data, calculate the intermediate frame image data, and replace the cached previous frame image data with the compressed image data; The intermediate frame image data and the compressed frame image data are output to the graphics display system in sequence.

[0070] Optionally, the processor 710 is also used for: When a touch event is detected under the condition that frame interpolation is enabled and the user is in the first usage scenario, image data is output according to the first frame rate. The first usage scenario includes at least one of the following: game scenario and video playback scenario.

[0071] Optionally, the first frame rate is the frame rate without compression.

[0072] It should be understood that, in this embodiment, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0073] The memory 709 can be used to store software programs and various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0074] Processor 710 may include one or more processing units; optionally, processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 710.

[0075] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image data processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0076] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0077] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image data processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0078] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0079] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image data processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0080] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0082] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image data processing method, characterized in that, include: When frame interpolation is enabled and a touch event is detected, image data is output at the first frame rate and the image data is cached. When frame interpolation is enabled and no touch event is detected, frame compression is performed to obtain image data with a second frame rate after frame compression. The image data after frame compression is cached, frame interpolation is performed on the image data after frame compression, and the image data after frame interpolation is output. The frame rate of the image data after frame interpolation is the first frame rate, and the second frame rate is less than the first frame rate.

2. The method according to claim 1, characterized in that, The method further includes: Upon detecting a touch event, the first time is set to the touch time corresponding to the touch event; If no touch event is detected, the first time is set to the first value; When frame interpolation is enabled and a touch event is detected, outputting image data according to the first frame rate includes: When frame interpolation is enabled, the first time is not equal to the first value, and the difference between the second time and the first time is less than the preset frame compression duration, image data is output according to the first frame rate. The step of performing frame compression processing when frame interpolation is enabled and no touch event is detected includes: When frame interpolation is enabled, the first time is equal to the first value, or the difference between the second time and the first time is greater than or equal to the preset frame compression duration, frame compression processing is performed. The second time is the refresh time of the image data.

3. The method according to claim 2, characterized in that, The preset frame compression duration is determined based on the frame compression rate, and the first value is 0.

4. The method according to claim 1, characterized in that, The step of performing frame interpolation on the compressed image data and outputting the interpolated image data includes: Retrieve the cached image data of the previous frame; Based on the previous frame image data and the compressed image data, calculate the intermediate frame image data, and replace the cached previous frame image data with the compressed image data; The intermediate frame image data and the compressed frame image data are output to the graphics display system in sequence.

5. The method according to any one of claims 1 to 4, characterized in that, When frame interpolation is enabled and a touch event is detected, outputting image data according to the first frame rate includes: When a touch event is detected under the condition that frame interpolation is enabled and the user is in the first usage scenario, image data is output according to the first frame rate. The first usage scenario includes at least one of the following: game scenario and video playback scenario.

6. An image data processing apparatus, characterized in that, include: The first processing module is used to output image data according to the first frame rate and cache the image data when the frame interpolation function is enabled and a touch event is detected; The second processing module is used to perform frame compression processing when the frame interpolation function is enabled and no touch event is detected, to obtain image data with a second frame rate after frame compression processing, to cache the image data after frame compression processing, to perform frame interpolation processing on the image data after frame compression processing, and to output the image data after frame interpolation processing, wherein the frame rate of the image data after frame interpolation processing is the first frame rate, and the second frame rate is less than the first frame rate.

7. The image data processing apparatus according to claim 6, characterized in that, The image data processing device further includes: The setting module is used to set the first time to the touch time corresponding to the touch event when a touch event is detected; and to set the first time to a first value when no touch event is detected. The first processing module is used to output image data according to the first frame rate when the frame interpolation function is enabled, the first time is not equal to the first value, and the difference between the second time and the first time is less than the preset frame compression duration. The second processing module is used to perform frame compression processing when the frame interpolation function is enabled, the first time is equal to the first value, or the difference between the second time and the first time is greater than or equal to the preset frame compression duration. The second time is the refresh time of the image data.

8. The image data processing apparatus according to claim 7, characterized in that, The preset frame compression duration is determined based on the frame compression rate, and the first value is 0.

9. The image data processing apparatus according to claim 6, characterized in that, The second processing module includes: The acquisition unit is used to acquire the cached image data of the previous frame; The calculation unit is used to calculate the intermediate frame image data based on the previous frame image data and the compressed frame image data, and replace the cached previous frame image data with the compressed frame image data. The output unit is used to output the intermediate frame image data and the compressed frame image data to the graphics display system sequentially.

10. The image data processing apparatus according to any one of claims 6 to 9, characterized in that, The first processing module is used to detect a touch event when the frame interpolation function is enabled and the application is in a first usage scenario, and output image data according to a first frame rate. The first usage scenario includes at least one of the following: game scenario and video playback scenario.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image data processing method as described in any one of claims 1 to 5.

12. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image data processing method as described in any one of claims 1 to 5.

Images