Data processing method and device, equipment and storage medium

Abstract

The invention provides a data processing method and device, equipment and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: when an obtained first real frame image and an obtained second real frame image meet a frame insertion condition, processing the first real frame image to obtain a first image and a mask image used for marking to-be-filled pixels and effective pixels in the first image, sampling mask data of the mask image pixel by pixel, and obtaining a second image; determining whether each pixel in the first image is a to-be-filled pixel, performing color filling on the to-be-filled pixel in the first image by using the color data of the first real frame image read from the memory, and generating a prediction frame image displayed between the first real frame image and the second real frame image; wherein the color data of the effective pixels in the first image is reserved. According to the technical scheme, the memory access times and the data volume of single memory access can be remarkably reduced, the power consumption of equipment is reduced, and the problem of heating of the equipment is solved.

Description

Data processing method, device, equipment and storage medium This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on August 31, 2022, with application number 202211064725.8 and application name “Data processing method, device, equipment and storage medium”, all contents of which are incorporated by reference in this application. Technical Field The present application relates to the field of data processing technology, and in particular to a data processing method, device, equipment and storage medium. Background Art With the continuous development of computer graphics, image rendering, as the most important research topic in computer graphics, has become an important technology in image processing. Frame prediction technology is mainly used to improve frame rate, reduce the sense of freeze in image rendering process and reduce the power consumption of game rendering. At present, in order to reduce the amount of data that needs to be processed during the image rendering process, texture compression technology is usually used. However, this texture compression technology can only process image texture data created offline. During the image rendering process, it is necessary to perform multiple image texture data reading and rendering target writing operations online, which will result in high power consumption and equipment heating problems. Summary of the invention The embodiments of the present application provide a data processing method, apparatus, device and storage medium to solve the problems of high power consumption and device heating in an image rendering process. The first aspect of the present application provides a data processing method, which can be executed by a graphics processor GPU in an electronic device. When the GPU receives a rendering instruction from a central processing unit CPU and performs a rendering operation, it can also execute the following data processing method. Optionally, the data processing method includes: the GPU obtains a first real frame image and a second real frame image, and when the interpolation condition is met between the first real frame image and the second real frame image, the GPU processes the first real frame image to obtain a first image and a mask image of the first image, the mask image is used to mark the pixels to be filled and the valid pixels in the first image, so that in the process of pixel filling of the first image, the mask data of the mask image is sampled pixel by pixel to determine whether each pixel in the first image is a pixel to be filled, and for any pixel to be filled in the first image, the color data of the first real frame image is read from the memory, and the pixel to be filled in the first image is filled with color using the color data, so as to generate a predicted frame image displayed between the first real frame image and the second real frame image; wherein the color data of the valid pixels in the first image is retained. In an embodiment of the present application, when an interpolation condition is met between two adjacent real frame images generated by a GPU, the first real frame image is processed to generate a first image having pixels to be filled and a mask image corresponding to the first image. In this way, in the process of pixel filling of the pixels to be filled in the first image, the pixels to be filled and the valid pixels in the first image can be determined by sampling the mask data in the mask image, so that the valid pixels are not processed, the color data of the valid pixels in the first image are retained, and for any pixel to be filled in the first image, the color data of the first real frame image is read from the memory, and the color data is used to color the pixels to be filled in the first image that are missing pixel data. Color filling can be performed to obtain a predicted frame image for display between the first real frame image and the second real frame image. The display frame rate of the image is improved by generating a predicted frame image for interpolation display. At the same time, during the image rendering process, only the pixels to be filled in the first image are filled with data, and the valid pixels in the first image are retained. The intermediate data of the pixel filling algorithm is reused. Under the premise of ensuring the color integrity of the predicted frame image, the number of memory accesses and the amount of data for a single memory access are significantly reduced, which effectively reduces the power consumption of the device and solves the problem of device heating. In a possible implementation of the first aspect, the interpolation condition includes that the motion amplitude between two consecutive real frame images is less than an amplitude threshold and that there is no semi-transparent object in the real frame image. By setting the interpolation condition, the GPU performs interpolation processing when the interpolation condition is met, thereby ensuring the quality of the generated predicted frame image, thereby improving the visual effect of the image. Optionally, after acquiring the first real frame image and the second real frame image, the method also includes: determining whether there is a semi-transparent object in the first real frame image; in response to the absence of a semi-transparent object in the first real frame image, calculating a first motion amplitude between the first real frame image and the second real frame image; determining whether the first motion amplitude is less than an amplitude threshold; in response to the first motion amplitude being less than the amplitude threshold, determining that an interpolation condition is met between the first real frame image and the second real frame image; in response to the presence of a semi-transparent object in the first real frame image and / or the first motion amplitude being greater than or equal to the amplitude threshold, determining that an interpolation condition is not met between the first real frame image and the second real frame image. It can be understood that in actual applications, the execution order of the two frame insertion conditions is not limited and can be determined according to the actual scenario. In this possible implementation, when there is no semi-transparent object in the first real frame image and the first motion amplitude between the first real frame image and the second real frame image is less than the amplitude threshold, it is determined that the interpolation condition is met between the first real frame image and the second real frame image, and when there is a semi-transparent object in the first real frame image and / or the first motion amplitude is greater than or equal to the amplitude threshold, it is determined that the interpolation condition is not met between the first real frame image and the second real frame image. In this way, the occurrence of poor quality prediction frame images can be effectively reduced or avoided, and the purpose of ensuring interpolation quality on the basis of improving the frame rate is achieved. In a possible implementation of the first aspect, the color data of the first real frame image is read from the memory, and the color data is used to fill the pixels to be filled in the first image, including: determining at least one candidate reference pixel of the pixel to be filled in the first image; determining a target reference pixel among the at least one candidate reference pixel according to the object category to which each candidate reference pixel belongs; determining the color data of the target reference pixel by reading the memory according to the relationship between the target reference pixel and the first real frame image; and filling the pixels to be filled with color based on the color data of the target reference pixel. In this possible implementation, in the process of color filling the pixels to be filled in the first image based on the color data of the effective pixels in the first real frame image, firstly, the candidate reference pixels are determined, then the target reference pixels are determined based on the object category to which the pixels belong, and the color data of the target reference pixels are read from the memory, thereby realizing the color filling of the pixels to be filled. That is, the color data of the target reference pixels are read from the memory, and the number of memory accesses and the amount of data for a single memory access are significantly reduced while ensuring the color integrity of the predicted frame image, thereby effectively reducing the power consumption of the device. Optionally, in the first image, determining the candidate reference pixel of the pixel to be filled includes: starting from the position where the pixel to be filled is located, randomly moving to a first position in the first image; if the pixel at the first position is not the pixel to be filled, determining the pixel at the first position as the candidate reference pixel for the pixel to be filled; if the pixel at the first position is the pixel to be filled, starting from the first position, continuing to move randomly in the first image until moving to a second position that is not the pixel to be filled, and determining the second position as the pixel to be filled In this embodiment, the candidate reference pixels of the pixels to be filled can be determined by random movement search, which lays a foundation for the subsequent determination of the target reference pixels. Optionally, the color filling of the pixel to be filled based on the color data of the target reference pixel includes: determining the effective color data of the target reference pixel according to the weight corresponding to the target reference pixel and the color data of the target reference pixel; accumulating the effective color data to the existing color data of the pixel to be filled until the number of color accumulations of the pixel to be filled is equal to a first threshold value, thereby obtaining the current color data of the pixel to be filled; dividing the current color data by the first threshold value to obtain the first color data of the pixel to be filled; and writing the first color data to the position of the pixel to be filled in the first image. The weight value based on the target reference pixel is defined based on the distance between the pixel to be filled and the target reference pixel, so that the effective color data of the target reference pixel can be accurately determined, and then the color data to be actually filled into the pixel to be filled can be obtained based on the number of color accumulations of the first threshold value, so that the color quality of the pixel to be filled can be improved, thereby ensuring the filling effect. In a possible implementation of the first aspect, the processing of the first real frame image to obtain a first image and a mask image of the first image includes: performing two image distortion operations on the first real frame image using a motion vector according to the object category to which each pixel in the first real frame image belongs, to generate the first image; and generating the mask image of the first image according to whether each pixel in the first image has color data. In this possible implementation, the pixels of static objects and the pixels of dynamic objects in the first real frame image are offset respectively, and the mask image of the first image is generated based on whether the pixels are covered in the first image after the two offsets, that is, whether there is color data. The mask image generated in this way can accurately mark the pixels to be filled and the valid pixels in the first image. In a possible implementation of the first aspect, the processing of the first real frame image to obtain the first image and the mask image of the first image includes: generating a template cache of the first real frame image according to the category of the object to which each pixel in the first real frame image belongs; querying the template cache to determine the category of the object to which each pixel in the mask texture image belongs, the mask texture image is an image whose initial values ​​are all first values, and the first value is used to mark the pixels to be filled in the first image; according to the category of the object to which each pixel belongs, using a motion vector to perform two image distortion operations on the first real frame image and the mask texture image respectively to generate the first image and the mask image of the first image. In this possible implementation, the mask image is generated by displacing the mask texture image whose initial values ​​are all first values ​​according to the category of the object to which each pixel in the first real frame image belongs, which is the same as the displacement process of the first image, so that the mask image can accurately mark the pixels to be filled and the valid pixels in the first image. Optionally, performing two image warping operations on the first real frame image and the mask texture map respectively using a motion vector according to the object category to which each pixel belongs, to generate a first image and a mask image of the first image, includes: The multi-rendering target MRT technology is used to simultaneously perform the first image distortion operation corresponding to the motion vector on the pixels belonging to the first object category in the first real frame image and the mask texture map, so as to generate the distorted first real frame image and the distorted mask texture map; the second image distortion operation corresponding to the motion vector is simultaneously performed on the pixels belonging to the second object category in the distorted first real frame image and the distorted mask texture map, so as to generate the first image and the mask image of the first image; wherein the first object category is a static object and the second object category is a dynamic object, or the first object category is a dynamic object and the second object category is a static object. When the first real frame image is moved by the motion vector to generate the rendering data of the first image and the mask image, the MRT technology is used to simultaneously store the data in different memory areas, so as to simultaneously generate the first image and the mask image corresponding to the first image, thereby improving the image generation efficiency and thus improving the data processing efficiency in the image rendering process. In a possible implementation of the first aspect, sampling the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled includes: sampling the mask data of each pixel in the mask image pixel by pixel; in the mask image, if the mask data of the sampled pixel is equal to a first value, determining that the pixel at the corresponding position in the first image is a pixel to be filled; if the mask data of the sampled pixel is equal to a second value, determining that the pixel at the corresponding position in the first image is not a pixel to be filled. By sampling the mask data of the pixel in the mask image, it can be determined whether the pixel at the corresponding position in the first image is a pixel to be filled according to the value of the mask data, thereby laying a foundation for subsequent targeted pixel filling. Optionally, the mask image is a binary image, the first value is equal to 1, and the second value is equal to 0. When the mask data of the pixels in the mask image are represented by 1 and 0, the mask image is a single-channel 8-bit mask image, and for pixels that do not need to be filled on the first image, only one read operation is required for the 8-bit Mask data, rather than one read and one write operation for the 32-bit image data, thereby reducing the amount of data accessed. A second aspect of the present application provides a data processing device, which can be implemented by a graphics processor GPU in an electronic device. Optionally, the data processing device includes: An acquisition unit, used for acquiring a first real frame image and a second real frame image; a processing unit, configured to process the first real frame image when an interpolation condition is satisfied between the first real frame image and the second real frame image, to obtain a first image and a mask image of the first image, wherein the mask image is used to mark pixels to be filled and valid pixels in the first image; The processing unit is further used to sample the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled; A filling unit is used to read the color data of the first real frame image from the memory for any pixel to be filled in the first image, and use the color data to fill the pixel to be filled in the first image, so as to generate a predicted frame image displayed between the first real frame image and the second real frame image; wherein the color data of the valid pixels in the first image are retained. In a possible implementation manner of the second aspect, the interpolation condition includes that a motion amplitude between two consecutive real frame images is less than an amplitude threshold and that there is no semi-transparent object in the real frame image. Optionally, after the acquisition unit acquires the first real frame image and the second real frame image, the processing unit is further used to perform the following operations: determine whether there is a semi-transparent object in the first real frame image; in response to the absence of a semi-transparent object in the first real frame image, calculate the first motion amplitude between the first real frame image and the second real frame image; determine whether the first motion amplitude is less than an amplitude threshold; in response to the first motion amplitude being less than the amplitude threshold, determine that an interpolation condition is met between the first real frame image and the second real frame image; in response to the presence of a semi-transparent object in the first real frame image and / or the first motion amplitude is greater than or equal to the amplitude threshold, determine that the interpolation condition is not met between the first real frame image and the second real frame image. In a possible implementation manner of the second aspect, when the color data of the first real frame image is read from the memory and the color data is used to fill the pixel to be filled in the first image with color, the filling unit is specifically used to: determine at least one candidate reference pixel of the pixel to be filled in the first image; determine a target reference pixel in the at least one candidate reference pixel according to the object category to which each candidate reference pixel belongs; and determine the color data of the target reference pixel by reading the memory according to the relationship between the target reference pixel and the first real frame image; The pixel to be filled is filled with color based on the color data of the target reference pixel. Optionally, in the first image, when determining the candidate reference pixel of the pixel to be filled, the filling unit is specifically used to: take the position of the pixel to be filled as the starting point, and randomly move to the first position in the first image; if the pixel at the first position is not the pixel to be filled, determine the pixel at the first position as the candidate reference pixel for the pixel to be filled; if the pixel at the first position is the pixel to be filled, take the first position as the starting point, and continue to move randomly in the first image until moving to a second position that is not the pixel to be filled, and determine the second position as the candidate reference pixel for the pixel to be filled. Optionally, when filling the pixel to be filled with color based on the color data of the target reference pixel, the filling unit is specifically used to: determine the effective color data of the target reference pixel according to the weight corresponding to the target reference pixel and the color data of the target reference pixel; accumulate the effective color data to the existing color data of the pixel to be filled until the number of color accumulations of the pixel to be filled is equal to a first threshold, to obtain the current color data of the pixel to be filled; divide the current color data by the first threshold to obtain the first color data of the pixel to be filled; and write the first color data into the position of the pixel to be filled in the first image. In a possible implementation of the second aspect, when processing the first real frame image to obtain a first image and a mask image of the first image, the processing unit is specifically used to: perform two image warping operations on the first real frame image using a motion vector according to the object category to which each pixel in the first real frame image belongs to generate a first image; and generate a mask image of the first image according to whether there is color data for each pixel in the first image. In a possible implementation of the second aspect, when processing the first real frame image to obtain a first image and a mask image of the first image, the processing unit is specifically used to: generate a template cache of the first real frame image according to the category of the object to which each pixel in the first real frame image belongs; query the template cache to determine the object category to which each pixel in the mask texture map belongs, the mask texture map being an image whose initial values ​​are all first values, and the first value is used to mark the pixels to be filled in the first image; according to the object category to which each pixel belongs, use a motion vector to perform two image warping operations on the first real frame image and the mask texture map, respectively, to generate a first image and a mask image of the first image. Optionally, when performing two image warping operations on the first real frame image and the mask texture map respectively using a motion vector according to the object category to which each pixel belongs to generate a first image and a mask image of the first image, the processing unit is specifically used to: use multi-rendering target MRT technology to simultaneously perform a first image warping operation corresponding to the motion vector on the pixels in the first real frame image and the mask texture map belonging to the first object category to generate a distorted first real frame image and a distorted mask texture map; simultaneously perform a second image warping operation corresponding to the motion vector on the pixels in the distorted first real frame image and the distorted mask texture map belonging to the second object category to generate the first image and the mask image of the first image; wherein the first object category is a static object and the second object category is a dynamic object, or the first object category is a dynamic object and the second object category is a static object. In a possible implementation of the second aspect, when sampling the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled, the processing unit is specifically used to: sample the mask data of each pixel in the mask image pixel by pixel; in the mask image, if the mask data of the sampled pixel is equal to a first value, determine that the pixel at the corresponding position in the first image is a pixel to be filled; if the mask data of the sampled pixel is equal to a second value, determine that the pixel at the corresponding position in the first image is not a pixel to be filled. Optionally, the mask image is a binary image, the first value is equal to 1, and the second value is equal to 0. In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to A computer program is stored, and the processor is used to execute the computer program to perform the method described in the first aspect or any possible implementation manner of the first aspect. In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect. In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect. In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc. In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.). It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS FIG1 is a schematic diagram of the image rendering process; FIG2 is a schematic diagram of a structure of an electronic device; FIG3 is a software structure block diagram of an electronic device according to an embodiment of the present application; FIG4 is a flow chart of a data processing method provided in an embodiment of the present application; FIG5 is a schematic diagram of a process of generating a mask image; FIG6 is another schematic diagram of a process of generating a mask image; FIG7 is a flowchart of the embodiment shown in FIG6; FIG8 is a schematic diagram of an implementation of a predicted frame image generation solution in the related art; FIG9 is a schematic diagram of an implementation of a predicted frame image generation solution in an embodiment of the present application; FIG10 is a flow chart of a second embodiment of a data processing method according to an embodiment of the present application; FIG11 is a flow chart of a third embodiment of a data processing method according to an embodiment of the present application; FIG12 is a schematic diagram of a process for determining candidate reference pixels; FIG13 is a result schematic diagram of a data processing device provided in an embodiment of the present application; FIG. 14 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below: 1. Rendering Rendering refers to the process of software generating images from models. Among them, the model can be understood as the three-dimensional spatial representation of objects, which is a three-dimensional object or virtual scene strictly defined by language or data structure in the picture to be rendered, and it includes geometry, viewpoint, texture, lighting and other information. It can be understood that the model is a kind of 3D information, the image is a kind of 2D information, and the object in the image is a pixel range covered by the projection of the 3D model onto the 2D image. 2. Texture Data Texture data refers to data that can be recognized and processed by a graphics card. Texture data may be in a variety of formats, for example, the texture data may be in RGB format, RGBA format, DirectX texture compression (Direct Draw Surface, DDS) format, and the like. Image texture data refers to: texture data that has not been compressed and encoded, that is, the data format of the image texture data is a texture format. The data size of the image texture data is used to indicate the number of pixels included in the image texture data; for example, if the data size of the image texture data is 128×128, then the data size indicates that the image texture data includes 128×128 pixels. 3. Data to be rendered The data to be rendered includes model data and material data. Model data refers to the coordinate data, position coordinates, normal coordinates, texture coordinates, tangent coordinates, etc. of each triangle vertex in the model composed of a triangular mesh; if the three-dimensional scene contains animation, the model data also includes animation information. Material data can include highlight maps, material map physical color maps, reflectivity maps (the ratio of reflection and refraction on the surface of the material), etc., that is, material data saves data in the form of maps. 4. Rendering data The rendering data may include color data and brightness data, that is, data generated during the rendering process of the data to be rendered. 5. Prediction frame In the embodiments of the present application, the predicted frame is also called the predicted frame image, and the two are interchangeable. In the field of video and games, the predicted frame image is the main means to improve the frame rate. Among them, the generation method of the predicted frame image is: first, the motion vector between the two real frame images before and after is calculated, and the real frame image is displaced by pixels or pixel blocks based on the motion vector, so as to generate the predicted frame image. 6. Mask image The mask image refers to an image generated by performing occlusion processing on an image (entire or partial), and the mask image can be used to mark the area to be processed in the image. 7. Camera In this embodiment, the camera is a virtual camera used in a graphics rendering scene, through which a three-dimensional scene can be observed from a certain angle and direction, thereby obtaining an appearance display of the virtual scene. Generally, by changing the angle and direction of the virtual camera, different appearance displays of the virtual scene can be seen. 8. Other terms In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different. It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way. In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple. With the continuous development of computer graphics, rendering, as the most important research topic in computer graphics, has gradually been applied to various fields such as computers, videos, games, movies or TV special effects, and visual design. When using electronic devices for applications such as games, modeling, and home design that require the display of large and complex images, the electronic devices need to complete a large amount of rendering processing before displaying the images. Exemplarily, FIG1 is a schematic diagram of the image rendering process. As shown in FIG1, in an electronic device, when an application (such as a game application) needs to display an image, a rendering command for the image can be issued to the central processing unit (CPU) of the electronic device, and the rendering instruction includes the data to be rendered. The CPU can transmit the data to be rendered to the graphics processing unit (GPU) of the electronic device according to these rendering commands, and then control the GPU to render the image to obtain rendering data, which can be stored in the memory of the electronic device. When the electronic device needs to display a certain image, the GPU can read the rendering data of the image from the memory and control the display screen to display the image. In applications related to graphics rendering, such as games, virtual simulation, virtual reality (VR) / augmented reality (AR), etc., electronic devices usually need to process a large amount of texture data during the image rendering process. Texture data is a type of storage data and occupies a large amount of memory. However, in the case of games and other applications that require a high refresh rate for the display screen, the display screen in the electronic device has the characteristic of a high refresh rate. In the case of a high refresh rate, high-resolution texture data will cause high rendering power consumption of the electronic device, and it is easy to cause drawing jams when drawing images, resulting in the problem of image jams. In order to reduce image stuttering and reduce the power consumption of picture rendering, the prediction frame technology came into being. The prediction frame image is a prediction frame image generated between two real frame images. Inserting it between two real frames can increase the frame rate of the displayed image, thereby reducing the sense of stuttering and reducing the power consumption of game rendering. Since the prediction frame image is an image that needs to be inserted between two real frames for display, its display cannot affect the image quality, that is, it cannot violate the actual scene when displayed to the user. Therefore, during the rendering process, before generating the image prediction frame, the GPU determines that it can be inserted before executing the prediction frame image generation process. In the process of calculating the predicted frame image, the GPU needs to perform multiple image texture data reading and rendering target writing operations, and memory operations are relatively frequent. Moreover, in the image rendering process on the mobile terminal, due to the bandwidth transmission capacity limitations of the mobile terminal, frequent image texture data sampling will cause the device to heat up. Therefore, in order to reduce power consumption and reduce device heating, memory optimization technology has been proposed to reduce the power consumption load of the chip, reduce device heating and increase the device's usage time. At present, in order to reduce the amount of data that needs to be processed during the image rendering process, texture compression technology is usually used, that is, the image texture data created during the offline process is first compressed, and then the image texture data is read when the image texture data is needed to reduce the number of memory accesses. However, this texture compression technology can usually only process the image texture data created offline, and cannot process the calculation process data of the predicted frame image. It is still necessary to perform multiple image texture data reading and rendering target writing operations, which will result in high power consumption and equipment heating problems, seriously affecting the service life of the equipment. In response to the above technical problems, the conceiving process of the technical solution of this application is as follows: the inventors have found in practice that sometimes the motion amplitude between two real frames is too large or there are translucent objects in the real frame image, and the interpolation effect is not good at this time. Therefore, the GPU first determines whether it can be interpolated before interpolation; moreover, in the process of generating predicted frame images in the game field, because there are multiple texture reading and rendering target writing operations, memory operations are relatively frequent. In practical applications, the rotation of the virtual camera will cause pixel loss in the edge area of ​​the predicted frame image, and the pixel offset will cause blank areas in the predicted frame image. Therefore, when reading image rendering data from the memory and writing the predicted frame image in the traditional way, there will be duplicate data copies. Therefore, an algorithm can be designed to mark the duplicated image area data and optimize the subsequent processing algorithm to achieve the purpose of reducing the amount of memory access data. Based on the above technical conception process, an embodiment of the present application provides a data processing method. After acquiring two real frame images, it is first determined whether the interpolation conditions are met. If so, the process of generating a predicted frame image is executed. That is, when processing the first real frame image, in addition to generating a predicted frame image with a pixel area to be filled, a mask image can also be generated. The mask image is used to mark the blank pixel area (i.e., the pixel area to be filled) and the valid pixel area (the pixel area that does not require operation) in the predicted frame image. In the pixel filling stage, based on the mask image, it can be determined whether to fill data for each pixel in the predicted frame image. If so, the specific operation of pixel filling is performed. If not, the pixel is not filled with data. This technical solution, in the process of generating a predicted frame image, uses a mask image to mark the image areas that need to be processed and those that do not need to be processed, and can reuse the rendering data of the real frame image, that is, the intermediate data of the data processing method, without performing any operation on the image area where the rendering data exists, thereby significantly reducing the number of memory accesses and the amount of data in a single memory access, and effectively reducing the power consumption of data processing and the heating problem of the equipment. Moreover, in this solution, the display frame rate can be improved by generating a predicted frame image for interpolated display. At the same time, since the inserted frame, that is, the predicted frame image, is obtained by predicting the motion vector and does not need to be obtained through rendering, the rendering workload is small and the delay is short. The solution provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings. It should be noted that the data processing method provided in the embodiment of the present application can be applied to electronic devices. The terminal device in the embodiment of the present application can be a terminal device in any form, for example, the terminal device can include a handheld device with image processing function, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (PLANs). The embodiments of the present application do not limit this. As an example but not a limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be called wearable smart devices, which are a general term for wearable devices that are designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. A wearable device is not just a hardware device, but also achieves powerful functions through software support, data interaction, and cloud interaction. In a broad sense, wearable smart devices include functional It is full-featured, large in size, and can realize complete or partial functions without relying on smartphones, such as smart watches or smart glasses, etc., as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring. In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things. The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc. In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. It is understandable that the embodiments of the present application do not impose any special restrictions on the specific form of the electronic device. In some embodiments, the electronic device may have a display function. For example, the electronic device may render an image according to a rendering command issued by an application program, and display the rendered image to the user. In order to better understand the embodiments of the present application, the structure of the electronic device in the embodiments of the present application is introduced below: Fig. 2 is a schematic diagram of a structure of an electronic device. As shown in Fig. 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution. The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiver / transmitter (universal asynchronous receiver / transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input / output (general-purpose input / output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc. It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods. The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information. The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), a low temperature polycrystalline oxide (LTPO), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor. The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, data files such as music, photos, and videos are stored in the external memory card. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like. The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100. It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware. It is worth noting that the product implementation form of the embodiment of the present application is a program code contained in the machine learning and deep learning platform software and deployed on the electronic device. The program code of the embodiment of the present application can be stored inside the electronic device. When running, the program code runs in the host memory and / or GPU memory of the electronic device. In practical applications, the software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device. Exemplarily, FIG3 is a software structure block diagram of an electronic device of an embodiment of the present application. As shown in FIG3, the layered architecture of the electronic device divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages. As shown in FIG3 , the application layer may include applications such as camera, gallery, calendar, call, map, short message, navigation, Bluetooth, music, video, and settings. The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. As shown in FIG3 , the application framework layer may include an input manager service (IMS). Of course, the application framework layer may also include a display policy service, a power management service (PMS), a display management service (DMS), an activity manager, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc., and the present application embodiment does not impose any limitation on this. Android runtime includes core library and virtual machine. Android runtime is responsible for scheduling and management of Android system. Core library consists of two parts: one is the function that Java language needs to call, and the other is Android core library. The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. The system library may include multiple functional modules, such as status monitoring service, surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. The status monitoring service is used to determine the specific orientation of the mobile phone, the physical state of the flexible screen, etc. according to the monitoring data reported by the kernel layer. The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing and layer processing. It is a drawing engine for 2D drawing. The kernel layer is a layer between hardware and software. The kernel layer at least includes display driver, sensor driver, TP driver, camera driver, audio driver, etc., and the present application embodiment does not impose any restrictions on this. Optionally, as shown in Figure 3, the system library and kernel layer below the application framework layer can be called the underlying system, and the underlying system includes an underlying display system for providing display services. For example, the underlying display system includes a display driver in the kernel layer and a surface manager in the system library. In conjunction with the schematic diagrams shown in Figures 1 to 3, the technical solution of the present application is described in detail through specific embodiments, that is, the process of generating a predicted frame image in the image rendering process is explained. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. It can be understood that in this embodiment, referring to the schematic diagram described in Figure 1 above, the rendering data of a real frame image is stored in the memory, and the GPU can execute the technical solution provided in the embodiment of the present application to generate a predicted frame image based on the real frame image and the rendering data of the real frame image stored in the memory. For example, FIG4 is a flow chart of a data processing method provided in an embodiment of the present application. This embodiment is explained by taking a GPU in an electronic device as the execution subject. As shown in FIG4, in this embodiment, the data processing method may include the following steps: S401, acquiring a first real frame image and a second real frame image. The first real frame image and the second real frame image are images generated after the GPU performs a rendering operation based on the acquired data to be rendered. During the image rendering process, the rendering data of the first real frame image and the second real frame image are stored in the memory. It can be understood that the data to be rendered is the data in the rendering command sent by the target application to the CPU, and is the data input from the CPU to the GPU during the image rendering process. For example, the target application can be a game application, a virtual simulation application, or other related application that requires a high refresh rate display. In practical applications, the first real frame image and the second real frame image can be any two consecutive real frame images in multiple frames of images generated by GPU based on the data to be rendered. Usually, these real frame images are generated in time sequence, and different frame images may include different numbers of rendering objects. Therefore, the rendering data of multiple frames of real frame images can be stored in the memory, and the embodiments of the present application do not limit this. S402: When an interpolation condition is satisfied between the first real frame image and the second real frame image, the first real frame image is processed to obtain a first image and a mask image of the first image. The mask image is used to mark the pixels to be filled and the valid pixels in the first image. Exemplarily, in the mask image, the first value is used to mark the pixels to be filled in the first image, and the second value is used to mark the valid pixels in the first image. In an embodiment of the present application, after the GPU generates rendering data of a real frame image based on a rendering command of the CPU, it can determine whether an interpolation condition is met between two consecutive real frame images based on preset configuration information. If so, a determined motion vector is used to move pixels or pixel blocks of the acquired first real frame image to generate a first image; if not, the technical solution of the present application is not executed, and the rendering process of the real frame image is directly executed. Optionally, the interpolation condition may include that the motion amplitude between two consecutive real frame images is less than a preset amplitude threshold and there is no semi-transparent object in the real frame image. That is, if the motion amplitude between two real frame images is greater than or equal to the preset amplitude threshold, there is too much missing pixel information between two adjacent real frames, and the missing pixel information cannot be restored through the two real frame images. If interpolation is performed at this time, the generated predicted frame image will be inconsistent with the previous and next real frame images, resulting in a poor user visual experience. If there are semi-transparent objects in the real frame image, processing the A channel data of the pixels at this time will damage the visual effect of the original image, so interpolation cannot be performed. It can be understood that whether there is a semi-transparent object in the real frame image can be understood as whether there is an object with a transparency greater than a transparency threshold in the real frame image. The transparency threshold can be preset in the GPU by the user based on the actual scene, which is not described here. The object in the image can also be interpreted as an object, etc., which is not limited in this embodiment. In an embodiment of the present application, after acquiring the first real frame image and the second real frame image, the GPU may first determine whether there is a semi-transparent object in the first real frame image. If so, it is determined that the interpolation condition is not satisfied between the first real frame image and the second real frame image; if not, the first motion amplitude between the first real frame image and the second real frame image is calculated, and it is determined whether the first motion amplitude is less than the amplitude threshold; if so, it is determined that the interpolation condition is satisfied between the first real frame image and the second real frame image; if not, it is determined that the interpolation condition is not satisfied between the first real frame image and the second real frame image. It can be understood that the embodiment of the present application does not limit the order of judging the two conditions, that is, the GPU can also first calculate the first motion amplitude between the first real frame image and the second real frame image, and determine whether the first motion amplitude is less than the amplitude threshold, and then determine whether there is a semi-transparent object in the real frame image. It can be determined according to the actual scene and will not be elaborated here. That is, when there is no semi-transparent object in the first real frame image and the first motion amplitude between the first real frame image and the second real frame image is less than the amplitude threshold, it is determined that the interpolation condition is met between the first real frame image and the second real frame image; when there is a semi-transparent object in the first real frame image and / or the first motion amplitude between the first real frame image and the second real frame image is greater than or equal to the amplitude threshold, it is determined that the interpolation condition is not met between the first real frame image and the second real frame image. In an embodiment of the present application, by setting the interpolation condition, the GPU performs interpolation processing when the interpolation condition is met, thereby ensuring the quality of the generated predicted frame image, thereby improving the visual effect of the image. In an embodiment of the present application, a motion vector may be used to move pixels or pixel blocks of the acquired first real frame image, thereby generating a first image. In an embodiment of the present application, the first image is an image having a pixel area to be filled. It can be understood that the motion vector may be obtained by estimating the motion vector between two adjacent real frame images, or may be determined based on the motion trajectory of the object in the first real frame image, and the embodiments of the present application do not limit this. Optionally, the pixel area to be filled in the first image may include an edge area of ​​the predicted frame image and a blank area of ​​the predicted frame image. The edge area of ​​the predicted frame image is usually caused by the rotation of the virtual camera, and the blank area of ​​the predicted frame image is usually caused by pixel offset. The embodiment of the present application does not limit the specific type of the pixel area to be filled. In an embodiment of the present application, when processing the first real frame image or processing the first image, a mask image of the first image can also be obtained. The mask image can be a single-channel 8-bit Mask map, which is used to mark the edge area of ​​the predicted frame image in the first image and the blank area inside the image that needs to be filled with pixels. That is, the mask image is used to mark the pixel area to be filled in the first image. It can be understood that in this embodiment, the edge area of ​​the prediction frame image and the blank area inside the prediction frame image that need to be filled with pixels are collectively referred to as pixel areas to be filled. In this embodiment, the mask image of the first image can be generated in different ways, and the mask image and the first image The images have the same pixels and the pixel positions are the same. In a possible implementation of the embodiment of the present application, the first image can be generated by performing two image warping operations on the first real frame image using a motion vector according to the object category to which each pixel in the first real frame image belongs, and then a mask image of the first image is generated based on whether color data exists for each pixel in the first image. For example, Fig. 5 is a schematic diagram of a process of generating a mask image. As shown in Fig. 5, firstly, a motion vector is processed on a first real frame image to generate a first image, and then a mask image of the first image is determined based on whether there is color data for each pixel in the first image, that is, whether the pixel is an edge pixel or a blank pixel. It can be understood that the absence of color data in the pixels in the first image can be understood as that after a pixel shift is performed on the pixels covered by the dynamic object based on the motion vector and another pixel shift is performed on the pixels covered by the static object, both shift results do not cover the pixels so that there is no color data. In a possible implementation of the embodiment of the present application, the first real frame image can be processed by using multiple render targets (MRT) technology to simultaneously generate the first image and the mask image of the first image. For example, after the first real frame image is moved by pixels or pixel blocks using a motion vector, the first image and the mask image of the first image are generated by using the MRT technology. Exemplarily, according to the category of the object to which each pixel in the first real frame image belongs, a template cache of the first real frame image is generated, the template cache is queried to determine the category of the object to which each pixel in the mask texture image belongs, and then based on the MRT technology, according to the category of the object to which each pixel belongs, a motion vector is used to perform two image warping operations on the first real frame image and the mask texture map respectively, and a first image and a mask image of the first image are generated simultaneously. Exemplarily, FIG6 is another schematic diagram of the process of generating a mask image. As shown in FIG6, when the first real frame image is moved by pixels or pixel blocks using motion vectors to generate the rendering data of the first image and the mask image, the MRT technology can be used to simultaneously place data in different memory areas to simultaneously generate multiple different targets. Therefore, in this embodiment, in the process of performing an image distortion operation on the first real frame image using motion vectors, the MRT technology can be used to write the image data after the image distortion to different rendering targets respectively. For example, the mask image and the first image (the predicted frame image of the first real frame image) can be bound as rendering targets at the same time, so that the mask image and the first image with the pixel area to be filled can be generated at the same time. For example, the following explanation is given by taking the execution environment of the embodiment of the present application as openGL. It is understandable that the execution environment of the embodiment of the present application may also be a vulkan environment, which is not limited by the embodiment of the present application. Specifically, the principle of simultaneously generating the mask image and the first image based on the MRT technology is as follows: 1. The GPU creates multiple rendering buffers; for example, two rendering buffers are created, namely M1 and M2. The GPU can pass M1 and M2 as parameters to the current rendering process based on the first call instruction sent by the CPU, such as the glFramebufferTexture instruction. In this way, the rendering target of the current rendering process is bound to both the rendering buffer M1 and the rendering buffer M2, that is, data can be written to the rendering buffer M1 and the rendering buffer M2 at the same time; 2. The GPU can call the glDrawBuffers interface based on the CPU interface call instruction to specify the number of rendering buffers bound to the rendering target; for example, in this embodiment, it is necessary to generate the first image and the mask image at the same time, then the number of buffers bound to the rendering target is specified to be 2, for example, the above-mentioned rendering buffer M1 and rendering buffer M2. 3. In the algorithm of the pixel shader, since there are two rendering targets specified in this embodiment, the GPU defines two different outputs color1 and color2, where color1 is used to render and generate the first image, and color2 is used to render and generate the mask image of the first image; 4. The GPU can use the distorted data of the first real frame image based on the CPU's call drawing instruction glDrawIndex By rendering color1 and color2 at the same time, and writing the rendering result of color1 into the bound rendering buffer M1, and writing the rendering result of color2 into the bound rendering buffer M2, the purpose of outputting to the rendering buffer M1 and the rendering buffer M2 at the same time in one rendering process is achieved, thereby obtaining the mask image and the first image. S403 , sampling the mask data of the mask image pixel by pixel, and determining whether each pixel in the first image is a pixel to be filled. In an embodiment of the present application, when determining whether each pixel in the first image needs to be filled with data, it can be determined based on the mask data of the pixel at the corresponding position in the mask image. For example, the mask data of each pixel (i.e., at the pixel position) in the mask image can be sampled pixel by pixel to determine whether the pixel at the corresponding position in the first image is a pixel to be filled. In an optional implementation method, the mask data value of each pixel in the mask image can be sampled pixel by pixel. Then, in the mask image, if the mask data value of the first pixel is equal to a first value, the pixel in the first image at the same position as the first pixel is determined to be the pixel to be filled. If the mask data value of the first pixel is equal to a second value, the pixel in the first image at the same position as the first pixel is determined not to be the pixel to be filled. The first pixel is any pixel of the mask image. Exemplarily, the mask image can be a binary image, that is, the Mask data of each pixel in the mask image is a numerical value, and is one of two numerical values, and the value of the Mask data is used to indicate whether the pixel at the corresponding position in the first image needs to be filled with data. For example, in the mask image, the pixel area to be filled in the first image is marked by the Mask data being equal to the first numerical value, and the pixel area in the first image that does not need to perform the pixel filling operation is marked by the Mask data being equal to the second numerical value. The first numerical value and the second numerical value are different. As an example, the first value is equal to 1, and the second value is equal to 0. At this time, the device is preset with the following provisions: in the mask image, if the Mask data of a certain pixel is 1, the pixel at the corresponding position in the first image is determined to be a pixel to be filled, and if the Mask data of a certain pixel is 0, the pixel at the corresponding position in the first image is determined not to be a pixel to be filled. As another example, the first value is equal to 0, and the second value is equal to 1. At this time, the device is preset with the following provisions: in the mask image, if the Mask data of a certain pixel is 0, it is determined that the pixel at the corresponding position in the first image is a pixel to be filled, and if the Mask data of a certain pixel is 1, it is determined that the pixel at the corresponding position in the first image is not a pixel to be filled. It is understandable that the first value and the second value may also be represented by other values ​​or characters, for example, the first value is identified by character A, the second value is identified by character B, etc. The embodiment of the present application does not limit the specific implementation of the first value and the second value, which can be set based on actual needs and will not be described in detail here. As an example, if it is determined based on the Mask data at the sampled pixel that the pixel to be sampled does not need to perform a pixel filling operation, the rendering data at the corresponding position on the first image is retained. That is, after reading the mask data from the acquisition pixel in the mask image, it is determined that the pixel corresponding to the sampled pixel at the corresponding position in the first image is not the pixel to be filled, then the color data is no longer read from the memory, and the process of writing to the first image is not performed. It can be understood that since the mask image is a binary image, a single-channel 8-bit mask image, when there is no need to perform a pixel filling operation on the sampled pixels, the memory access overhead is only an 8-bit read operation. As another example, if the sample is determined to be a pixel to be filled based on the Mask data of the sampled pixel, a pixel filling operation is performed on the pixel at the corresponding position in the first image, that is, the color data of the pixel is obtained by utilizing the logical process and algorithm of pixel filling, and written to the pixel position in the first image. S404: for any pixel to be filled in the first image, read the color data of the first real frame image from the memory. The color data is used to fill the pixels to be filled in the first image with colors. S405: No processing is performed on any valid pixel in the first image, and the color data of the valid pixel is retained. That is, in the embodiment of the application, the color data of the valid pixels in the first image are retained. S406: When it is determined that each pixel in the first image has color data, generate a predicted frame image displayed between the first real frame image and the second real frame image. In an embodiment of the present application, in the pixel filling stage of an image, a first image after pixel or pixel block movement may be used as a target rendering image, where the first image is an image with a pixel area to be filled. Exemplarily, when determining the pixels to be filled in the first image, the target reference pixels of the pixels to be filled can be found by moving in the first image, and then the rendering data of the target reference pixels can be determined by reading the memory, in which the color data of the first real frame image is stored. The pixel filling algorithm is performed using the RGBA four-channel data to calculate the color data of the pixels to be filled, and the calculation results are written into the pixels at the corresponding positions in the first image. In an embodiment of the present application, after color filling of all the pixels to be filled in the first image based on the rendering data of the first real frame image, a color complete image can be obtained, that is, a predicted frame image that can be inserted between the first real frame image and the second real frame image for display, thereby improving the frame rate of the displayed image and reducing the sense of freeze in the picture. From the above analysis, it can be seen that the display frame rate of the image is improved by generating a predicted frame image for interpolated display. At the same time, during the image rendering process, only the pixels to be filled in the first image are filled with data, and the valid pixels in the first image are retained. The intermediate data of the pixel filling algorithm is reused. Under the premise of ensuring the color integrity of the predicted frame image, the number of memory accesses and the amount of data for a single memory access are significantly reduced, which effectively reduces the power consumption of the device and solves the problem of device heating. Based on the above steps, FIG7 is a flowchart of the embodiment shown in FIG6. The flowchart is explained by generating a predicted frame image of a real frame image of a game. In this embodiment, the real frame image of the game is a first real frame image, and the mask image is represented by a mask image. Among them, the mask data of a certain pixel in the mask image is equal to 0, indicating that the pixel at the corresponding position in the predicted frame image with the pixel to be filled is not the pixel to be filled; the mask data of a certain pixel in the mask image is equal to 1, indicating that the pixel at the corresponding position in the predicted frame image with the pixel to be filled is the pixel to be filled. As shown in FIG7 , for the real frame image of the game, the image distortion operation can be performed on the real frame image of the game through the motion vector to generate a mask image and a predicted frame image with pixels to be filled; then in the pixel filling stage, the predicted frame image with pixels to be filled is set as the rendering target, and then based on the mask data at the pixel position read from the mask image, the pixel area in the predicted frame image with pixels to be filled that needs to perform the pixel filling operation is determined, and finally the pixel area that needs to perform the pixel filling operation is filled with data to obtain the predicted frame image of the real frame image of the game. Optionally, if the mask data of a pixel is equal to 0, no operation is performed on the pixel at the corresponding position in the predicted frame image with the pixel to be filled, the color data at the pixel is retained, and the mask data of other pixels in the mask image are read, and then it is determined whether the pixel filling operation is performed on other pixels in the predicted frame image with the pixel to be filled. Optionally, if the mask data of a pixel is equal to 1, a pixel filling operation is performed on the pixel at the corresponding position in the predicted frame image with the pixel to be filled, and the generated color data is written to the corresponding position in the predicted frame image with the pixel to be filled, and then the values ​​at other pixel positions in the mask image are read to determine whether other pixels in the predicted frame image with the pixel to be filled need to perform pixel filling operations, until all the pixels to be filled in the predicted frame image with the pixel to be filled are processed. The following uses a specific example to compare the bandwidth usage of the predicted frame image generation scheme provided by the present application and the predicted frame image generation scheme in the related art. Assume that the size of the images processed in the present application (the first real frame image, the first image, the mask image, and the predicted frame image of the first real frame image) is 1584 (width) × 720 (height), in pixels; The first real frame image, the first image and the predicted frame image include 4-channel 8-bit RGBA data, the processing rate of the pixel filling operation is 60 (fps), and the mask image includes a single-channel 8-bit data represented by 0 or 1. Exemplarily, FIG8 is a schematic diagram of the implementation of the predicted frame image generation scheme in the related art. As shown in FIG8, the first real frame image is a real frame image, and the rendering data of each pixel is stored in the memory, and the rendering data includes color data. In this way, when the first image (the predicted frame image with the pixels to be filled) is generated based on the first real frame image, when the first image needs to be filled with pixels, the color data of each pixel is read from the memory in sequence and then rendered into the first image, that is, one read operation and one write operation are required to obtain the predicted frame image of the first real frame image. In the example shown in FIG8 , when pixel filling is performed on the first image, the required transmission bandwidth is 1584 (width) × 720 (height) × 4 (RGBA8) × 60 (fps) × 2 (read + write) = 547430400 B / s = 522 M / s. Among them, 1 M / s = 1024 KB = 1024 2 B / s. Exemplarily, Fig. 9 is a schematic diagram of the implementation of the predicted frame image generation scheme in the embodiment of the present application. As shown in Fig. 9, the first real frame image is a real frame image, the rendering data of each pixel is stored in the memory, and the rendering data includes color data, so that when the first image is pixel-filled based on the mask image of the first image (used to mark the pixel area to be filled in the first image), the mask value of each pixel in the mask image is first read to determine whether it is necessary to perform a pixel filling operation on the pixel at the corresponding position in the first image. As an example, if there is no need to perform a pixel filling operation on the pixel at the corresponding position, the pixel shading operation at the corresponding position is abandoned, and the operation of writing data to the rendering target is not performed. That is, only one read operation is required for the 8-bit Mask data, instead of one read and one write operation for the 32-bit image data, thereby reducing the amount of data accessed. As another example, if a pixel filling operation needs to be performed on the pixel at the corresponding position, the color data of the first real frame image is read from the memory according to the pixel filling method, the color data of the pixel is calculated and rendered to the pixel position in the first image, and the predicted frame image of the first real frame image is obtained. That is, for this part, a read operation is performed on the 8-bit Mask data, and then a read and a write operation is performed on the 32-bit image data. Optionally, in practical applications, in order to increase the frame rate and improve the image realism, the displacement between the real frame image and the predicted frame image is usually relatively small. For example, in the first image, assuming that the area ratio of the pixel area to be filled and the first image is equal to 1%, therefore, in this embodiment, when the first image is filled with pixels, the required transmission bandwidth is 1584 (width) × 720 (height) × 4 (RGBA8) × 60 (fps) × 0.01 (area ratio of the pixel area to be filled) × 2 (read + write) + 1584 (width) × 720 (height) × 60 (fps) × 1 (for each pixel in the mask image, a read operation is required on the 8-bit Mask data) = 73903104 B / s = 70 M / s. From the above analysis, it can be seen that in the embodiment of the present application, when the first real frame image and the second real frame image meet the interpolation condition, the first image and the mask image of the first image are obtained by processing the acquired first real frame image, and the mask image is used to mark the pixels to be filled and the valid pixels in the first image, and then the mask data of the mask image is sampled pixel by pixel to determine whether each pixel in the first image is a pixel to be filled, and finally for any pixel to be filled in the first image, the color data of the first real frame image is read from the memory, and the color data is used to fill the pixel to be filled in the first image, and a predicted frame image that can be inserted between the first real frame image and the second real frame image is generated; wherein, the valid pixels in the first image are not processed, and the color data of the valid pixels are retained. In this technical solution, since the mask image is a single-channel 8-bit mask image, for pixels on the first image that do not need to be filled, only one read operation is required for the 8-bit Mask data, instead of one read and one write operation for the 32-bit image data, thereby reducing the amount of data accessed. For example, based on the above embodiment, FIG10 is a flow chart of the second embodiment of the data processing method provided by the embodiment of the present application. As shown in FIG10 , the above S402 can be implemented by the following steps: S1001. Generate a template cache of a first real frame image according to the category of the object to which each pixel in the first real frame image belongs. Among them, in the template cache, the template value of the pixel corresponding to the dynamic object is equal to the identifier of the dynamic object (a positive integer), and the template value of the pixel corresponding to the static object is equal to 0. It can be understood that the template cache is also an image, and the dynamic object in the first real frame image is marked by a positive integer, and each pixel in the static object in the first real frame image is marked by a value of 0. In this embodiment, during the normal rendering process of the image, the template value of each pixel when filling can be set according to whether the object to which each pixel belongs in the first real frame image is a dynamic object or a static object. In an optional embodiment, if the object to which a certain pixel in the first real frame image belongs is a dynamic object, the template value of the pixel is determined to be the identification (ID) of the object to which the pixel belongs, and the template value of the pixel is output to the template cache; if the object to which a certain pixel in the first real frame image belongs is a static object, the template value of the pixel is determined to be 0, and 0 is output to the template cache. S1002: Generate a mask texture map whose initial values ​​are all first values. The first value is used to mark the pixels to be filled in the first image. Optionally, in practical applications, a mask texture map may be created first, and the values ​​of each pixel in the mask texture map may be initialized to be a first value, for example, the first value is equal to 1. S1003 . For each pixel in the mask texture image, query the same position of the template cache to determine the object category to which each pixel in the mask texture image belongs. In the process of generating the first image and the mask image, for each pixel in the mask texture image, firstly, based on the pixel position of the pixel, the template cache of the first real frame image is queried, and the data at the same pixel position is read to obtain the object category to which the pixel belongs, that is, whether the pixel is a pixel of a dynamic object or a pixel of a static object. S1004. According to the object category to which each pixel belongs, use a motion vector to perform two image warping operations on the first real frame image and the mask texture image respectively, to generate a first image and a mask image of the first image. In an embodiment of the present application, for static objects in the first real frame image, a first image warping operation is performed on the first real frame image and the mask texture map, and during the image warping operation, it is determined whether the current pixel is a static pixel. If so, 0 is output to the mask texture map. If not, no operation is performed, that is, the data of the corresponding pixel position in the corresponding mask texture map is kept at 1, and accordingly, a distorted first real frame image and a distorted mask texture map are generated. Optionally, for dynamic objects in the distorted first real frame image, a second image distortion operation is performed on the distorted first real frame image and the distorted mask texture map, and during the image distortion operation, it is determined whether the current pixel is a dynamic pixel. If the current pixel is a dynamic pixel, 0 is output to the distorted mask texture map, otherwise, no operation is performed. At this time, the pixel data of the mask texture map after two distortions remains at 1. Correspondingly, the first real frame image is subjected to two distortion operations to generate the first image, and correspondingly, the mask texture image is also subjected to two image distortion operations to generate the mask image of the first image. Among them, since the pixels in the pixel area to be filled belong neither to dynamic objects nor to static objects, no operation is performed on them in the mask texture map, and the data of 1 is retained, while other places are output as 0, thereby generating a mask image of the first image. In the embodiment of the present application, the first real frame image is generated according to the category of the object to which each pixel in the first real frame image belongs. The template cache of the real frame image can determine the object category to which each pixel in the mask texture map whose initial values ​​are all the first values ​​belongs based on the template cache, and then the motion vector can be used to perform two image distortion operations on the first real frame image and the mask texture map respectively according to the object category to which each pixel belongs, so as to generate the first image and the mask image of the first image. In this technical solution, a specific method for generating the mask image is given, which lays a foundation for reducing the number of memory accesses and the amount of memory accesses in the subsequent prediction frame image generation process. For example, based on the above embodiment, FIG11 is a flow chart of the third embodiment of the data processing method provided by the embodiment of the present application. As shown in FIG11 , the above S404 can be implemented by the following steps: S1101. Determine at least one candidate reference pixel for a pixel to be filled in a first image. In this embodiment, for the pixel to be filled in the first image, the position of the pixel to be filled is taken as the starting point, and the pixel is randomly moved to the first position in the first image; then, it is determined whether the first position belongs to the area to be filled, if the first position does not belong to the area of ​​the pixel to be filled, the pixel at the first position is determined as the candidate reference pixel of the pixel to be filled, if the first position belongs to the area of ​​the pixel to be filled, the pixel is taken as the starting point, and the pixel is randomly moved in the first image until it moves to the second position that does not belong to the area of ​​the pixel to be filled, and the second position is determined as the candidate reference pixel of the pixel to be filled. Similarly, other candidate reference pixels can be determined in this way. Exemplarily, FIG12 is a schematic diagram of the process of determining candidate reference pixels. As shown in FIG12, assuming that the coordinate point of the pixel to be filled is the starting point P_0, the coordinate point of the pixel to be filled is taken as the starting point P_0, and randomly moves in a direction D_0 for a step length Offset_0 (O_0) to the first position P_i, and then determines whether the first position P_i is located in the pixel area to be filled. If the first position P_i is located in the pixel area to be filled, for example, if the moved P_1 is still located in the pixel area to be filled, then continue to move randomly in one direction, for example, D_1, until it reaches a position that is not the pixel to be filled, that is, the position of a valid pixel, for example, P_2. The pixel at the position P_2 is the candidate reference pixel of the pixel to be filled. At the same time, mark the position coordinates P_2 of the candidate reference pixel and the direction D_1 of the movement. S1102. Determine a target reference pixel among the at least one candidate reference pixel according to the object category to which each candidate reference pixel belongs. In this embodiment, for any one of the at least one candidate reference pixel mentioned above, the semantic map corresponding to the position can be sampled according to the position of the candidate reference pixel, and then it is determined whether the candidate reference pixel is a pixel of the set object category. If so, the candidate reference pixel is determined as the target reference pixel; if not, the candidate reference pixel is discarded. The semantic map is a memory area set during the graphics rendering process, called the depth template cache. The depth template cache has two channels, one channel is used to save the depth of the scene, and the other channel is called the template, which is the location of the semantics. When calling the rendering command, for each object in the image, if an integer data is passed in the command glStencilFunc(), the integer data will be written to the depth template cache, that is, the template channel of the pixel block covered by the object, so that there is template data as the semantic map. In an optional implementation, since the pixel area to be filled is a blank pixel that appears after the image is shifted, it is an area belonging to a static object in practical applications, and therefore, in this embodiment, the set object category is a static object. Accordingly, it can be determined whether each candidate reference pixel is a pixel of a static object. If a candidate reference pixel is a pixel of a static object, the candidate reference pixel is determined to be the target reference pixel; if a candidate reference pixel is not a pixel of a static object, that is, if the candidate reference pixel is a pixel of a dynamic object, the candidate reference pixel is discarded. Optionally, in practical applications, if no reference pixel of a static object that meets the requirements is found after multiple iterations of step S1101, a candidate reference pixel of a dynamic object may be selected as the target reference pixel. The motion patterns of dynamic objects and static objects are different. Therefore, when processing frame prediction, dynamic objects and static objects need to be processed separately, which will not be elaborated here. S1103. According to the relationship between the target reference pixel and the first real frame image, determine the color data of the target reference pixel by reading the memory. The memory stores color data of the first real frame image. Optionally, in this embodiment, after the rendering data of the real frame image is generated, it will be stored in the memory for subsequent use, so the color data of the first real frame image will be stored in the memory. Therefore, when determining that the candidate reference pixel is the target reference pixel, the memory can be queried based on the position of the target reference pixel in the first image, and the color data of the target reference pixel can be determined from the rendering data of the first real frame image. S1104 . Determine valid color data of the pixel to be filled according to the weight corresponding to the target reference pixel and the color data of the target reference pixel. Optionally, in this embodiment, the weight corresponding to the target reference pixel can be determined based on the positional relationship between the target reference pixel and the pixel to be filled, and then the color data of the target reference pixel and the weight corresponding to the target reference pixel are summed to determine the effective color data of the target reference pixel. S1105 , accumulating the above-mentioned valid color data to the existing color data of the pixel to be filled, until the number of color accumulations of the pixel to be filled is equal to a first threshold, thereby obtaining the current color data of the pixel to be filled. S1106 . Divide the current color data by the first threshold to obtain first color data of the pixel to be filled. Optionally, after calculating the valid color data of the target reference pixel, it can be added to the existing color data of the pixel to be filled, and then starting from the position coordinate P_i of the target reference pixel, the set displacement is moved in the direction of the first probability to find other object pixels (also target reference pixels) belonging to the set object category, and the color data of the other object pixels are added to the existing color data. The first probability direction movement setting displacement may be moving Offset_i in the D_i direction or in the random direction D_j. This embodiment does not limit this. However, the probability of moving Offset_i in the D_i direction is higher than the probability of moving Offset_i in the random direction D_j. Optionally, repeat the above-mentioned process S1101 to S1105 for a set number of times t, that is, until the color accumulation times of the pixel to be filled is equal to the first threshold, to obtain the current color data of the pixel to be filled, and divide the current color data by the accumulation times t to obtain the first color data of the pixel to be filled. Exemplarily, in this embodiment, when taking the pixel of the static object as the target reference pixel and calculating the color data of the pixel to be filled, the color data of the target reference pixel and the weight corresponding to the target reference pixel can be accumulated to the existing color data of the pixel to be filled; then, taking the position coordinate P_i of the target reference pixel as the starting point, move in the direction of the first probability to find the pixel belonging to the static object, and accumulate the pixel data of the static object to the existing color data. It can be understood that the weight data of the target reference pixel is fixed, and is a fixed value generated by mathematical calculation. Taking the pixel of a static object as the target reference pixel as an example, the weight corresponding to the target reference pixel is obtained by taking the data in this array according to the number of steps moved by P_0. For example, taking the weight distribution corresponding to the following static object as an example, the sampling position of the target reference pixel moves 1 step compared to the starting position, then the corresponding weight of the target reference pixel is 1.898840. If the sampling position of the target reference pixel moves 10 steps compared to the starting position, then the corresponding weight is 0.144245. The weight distribution corresponding to static objects is as follows: const highp float Distance_Inv

[0128] =float

[0128] ( 1.898840,1.824470,1.104720,0.296970,0.219880,0.197224,0.179284,0.154482, 0.147665,0.144245,0.067793,0.054142,0.050809,0.045755,0.035451,0.030171,0.030158,0.029904,0.025308,0.024105,0.022307,0.021846,0.020519,0.020473,0.020024,0.018623,0.015612,0.014383,0.013974,0.013166,0.012510,0.007830,0.007438,0.007360,0.007279,0.007151,0.007144,0.006906,0.006578,0.006517,0.006394,0.006342,0.006172,0.005755,0.005722,0.005310,0.004307,0.004235,0.004145,0.004093,0.003782,0.003714,0.003468,0.003244,0.003197,0.003029,0.002935,0.002846,0.002817,0.002719,0.002698,0.002652,0.002646,0.002585,0.002569,0.002503,0.002254,0.002125,0.002078,0.002053,0.001956,0.001880,0.001837,0.001764,0.001710,0.001572,0.001550,0.001550,0.001535,0.001511,0.001496,0.001491,0.001481,0.001449,0.001378,0.001371,0.001361,0.001341,0.001328,0.001323,0.001119,0.001086,0.001064,0.001058,0.001038,0.000988,0.000983,0.000974,0.000959,0.000956,0.000956,0.000926,0.000901,0.000869,0.000864,0.000862,0.000858,0.000806,0.000806,0.000789,0.000784,0.000783,0.000780,0.000778,0.000772,0.000766,0.000736,0.000714,0.000708,0.000695,0.000687,0.000684,0.000682,0.000678,0.000670,0.000668,0.000667,0.000666. ). It is understandable that the embodiments of the present application do not limit the specific values ​​in the weight distribution, which can be selected according to the actual scenario and will not be elaborated here. Optionally, compared with the weight distribution of static objects, the weight of the corresponding position in the weight distribution of dynamic objects is usually smaller. The specific value can be determined according to actual conditions and is not limited here. S1107 . Write the first color data into the position of the pixel to be filled in the first image. In this step, after determining the color data of the pixels to be filled, the color data of the pixels to be filled can be written into the rendering target to cover the original data of the pixels at the corresponding position of the rendering target, that is, written to the position of the pixels to be filled. After all the pixels to be filled in the first image are filled with color data, a predicted frame image of the first real frame image can be generated. In an embodiment of the present application, at least one candidate reference pixel of a pixel to be filled is determined in a first image, a target reference pixel in the at least one candidate reference pixel is determined according to the object category to which each candidate reference pixel belongs, and then the color data of the target reference pixel is determined by reading a memory, in which the rendering data of the first real frame image is stored, and then the effective color data of the pixel to be filled is determined according to the weight corresponding to the target reference pixel and the color data of the target reference pixel, and the effective color data of the pixel to be filled is accumulated to the existing color data of the pixel to be filled until the number of color accumulations of the pixel to be filled is equal to the first threshold value, and the current color data of the pixel to be filled is obtained, and finally the current color data is divided by the first threshold value to obtain the first color data of the pixel to be filled, and the first color data is written to the position of the pixel to be filled in the first image. In this technical solution, the color data of the target reference pixel is read from the memory, and the intermediate data of the pixel filling algorithm is reused, which significantly reduces the number of memory accesses and the amount of data for a single memory access while ensuring the color integrity of the predicted frame image, and effectively reduces the power consumption of the device. The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that In combination with the method steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The embodiment of the present application can divide the functional modules of the device for implementing the data processing method according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, FIG13 is a result schematic diagram of a data processing device provided in an embodiment of the present application. The data processing device can be implemented by a graphics processor GPU in an electronic device. As shown in FIG13 , the data processing device includes: An acquiring unit 1301 is used to acquire a first real frame image and a second real frame image; The processing unit 1302 is used to process the first real frame image when an interpolation condition is met between the first real frame image and the second real frame image to obtain a first image and a mask image of the first image, wherein the mask image is used to mark pixels to be filled and valid pixels in the first image; The processing unit 1302 is further configured to sample the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled; The filling unit 1303 is used to read the color data of the first real frame image from the memory for any pixel to be filled in the first image, and use the color data to fill the pixel to be filled in the first image, so as to generate a predicted frame image displayed between the first real frame image and the second real frame image; wherein the color data of the valid pixels in the first image are retained. In a possible implementation of the embodiment of the present application, the interpolation condition includes that the motion amplitude between two consecutive real frame images is less than an amplitude threshold and there is no semi-transparent object in the real frame image. Optionally, after the acquiring unit 1301 acquires the first real frame image and the second real frame image, the processing unit 1302 is further configured to perform the following operations: Determining whether there is a semi-transparent object in the first real frame image; In response to the absence of a semi-transparent object in the first real frame image, calculating a first motion amplitude between the first real frame image and the second real frame image; determining whether the first motion amplitude is less than an amplitude threshold; In response to the first motion amplitude being less than an amplitude threshold, determining that an interpolation condition is satisfied between the first real frame image and the second real frame image; In response to the presence of a semi-transparent object in the first real frame image and / or the first motion amplitude being greater than or equal to an amplitude threshold, it is determined that an interpolation condition is not satisfied between the first real frame image and the second real frame image. In a possible implementation of the embodiment of the present application, when reading the color data of the first real frame image from the memory and using the color data to fill the pixels to be filled in the first image with colors, the filling unit 1303 is specifically used to: In the first image, determining at least one candidate reference pixel for the pixel to be filled; Determining a target reference pixel among the at least one candidate reference pixel according to the object category to which each candidate reference pixel belongs; According to the relationship between the target reference pixel and the first real frame image, the color data of the target reference pixel is determined by reading the memory; The pixel to be filled is filled with color based on the color data of the target reference pixel. Optionally, when determining, in the first image, candidate reference pixels for the pixels to be filled, the filling unit 1303 is specifically configured to: Taking the position of the pixel to be filled as the starting point, randomly move to the first position in the first image; If the pixel at the first position is not the pixel to be filled, determining the pixel at the first position as a candidate reference pixel of the pixel to be filled; If the pixel at the first position is a pixel to be filled, starting from the first position, continue to move randomly in the first image until moving to a second position that is not a pixel to be filled, and determine the second position as a candidate reference pixel for the pixel to be filled. Optionally, when color filling is performed on the pixel to be filled based on the color data of the target reference pixel, the filling unit 1303 is specifically configured to: Determining effective color data of the target reference pixel according to a weight corresponding to the target reference pixel and color data of the target reference pixel; Accumulating the valid color data to the existing color data of the pixel to be filled until the number of color accumulations of the pixel to be filled is equal to a first threshold, thereby obtaining the current color data of the pixel to be filled; Dividing the current color data by the first threshold value to obtain first color data of the pixel to be filled; The first color data is written into the position of the pixel to be filled in the first image. In a possible implementation manner of the embodiment of the present application, when processing the first real frame image to obtain the first image and the mask image of the first image, the processing unit 1302 is specifically configured to: According to the object category to which each pixel in the first real frame image belongs, using a motion vector to perform two image warping operations on the first real frame image to generate a first image; A mask image of the first image is generated according to whether color data exists for each pixel in the first image. In a possible implementation manner of the embodiment of the present application, when processing the first real frame image to obtain the first image and the mask image of the first image, the processing unit 1302 is specifically configured to: generating a template cache of the first real frame image according to the category of the object to which each pixel in the first real frame image belongs; Querying the template cache to determine the object category to which each pixel in the mask texture image belongs, the mask texture image being an image whose initial values ​​are all first values, the first value being used to mark pixels to be filled in the first image; According to the object category to which each pixel belongs, the first real frame image and the mask texture map are respectively subjected to two image distortion operations using a motion vector to generate a first image and a mask image of the first image. Optionally, when performing two image warping operations on the first real frame image and the mask texture map respectively using a motion vector according to the object category to which each pixel belongs to generate a first image and a mask image of the first image, the processing unit 1302 is specifically configured to: Using a multi-rendering target MRT technology, a first image warping operation corresponding to a motion vector is simultaneously performed on the first real frame image and the pixels in the mask texture map belonging to the first object category to generate a warped first real frame image and a warped mask texture map; The pixels belonging to the second object category in the distorted first real frame image and the distorted mask texture image are simultaneously performing a second image warping operation corresponding to the motion vector to generate a first image and a mask image of the first image; The first object category is a static object, and the second object category is a dynamic object, or the first object category is a dynamic object, and the second object category is a static object. In a possible implementation of the embodiment of the present application, when sampling the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled, the processing unit 1302 is specifically configured to: Sampling the mask data of each pixel in the mask image pixel by pixel; In the mask image, if the mask data of the sampled pixel is equal to a first value, determining that the pixel at the corresponding position in the first image is a pixel to be filled; If the mask data of the sampled pixel is equal to the second value, it is determined that the pixel at the corresponding position in the first image is not a pixel to be filled. Optionally, the mask image is a binary image, the first value is equal to 1, and the second value is equal to 0. The data processing device provided in the embodiment of the present application can be used to execute the data processing method shown in Figure 4 above. Its implementation principle and effective effects can be found in the records in the above embodiment and will not be repeated here. 14 is a schematic diagram of a chip structure provided by an embodiment of the present application. As shown in FIG14 , a chip 1400 includes one or more (including two) processors 1401 , a communication line 1402 , a communication interface 1403 and a memory 1404 . The processor 1401 is implemented by a GPU, or the processor 1401 is implemented by a CPU and a GPU. In some implementations, the memory 1404 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof. The method described in the above embodiment of the present application can be applied to the processor 1401, or implemented by the processor 1401. The processor 1401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 1401 or an instruction in the form of software. The processor 1401 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application. The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 1404, and the processor 1401 reads the information in the memory 1404 and completes the steps of the above method in combination with its hardware. The processor 1401 , the memory 1404 , and the communication interface 1403 may communicate with each other via the communication line 1402 . In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software. The embodiments of the present application also provide a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiments of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer-readable storage The medium may be any available medium that can be stored by a computer or a data storage device such as a server, a data center, etc. that includes one or more available media integrated therein. For example, the available medium may include a magnetic medium (e.g., a floppy disk, a hard disk, or a tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). The embodiments of the present application also provide a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer. In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media. The present application embodiment is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the present application embodiment. It should be understood that each process and / or box in the flowchart and / or block diagram and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram. The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: Acquire a first real frame image and a second real frame image; When an interpolation condition is satisfied between the first real frame image and the second real frame image, the first real frame image is processed to obtain a first image and a mask image of the first image, wherein the mask image is used to mark pixels to be filled and valid pixels in the first image; Sampling the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled; For any pixel to be filled in the first image, the color data of the first real frame image is read from the memory, and the color data is used to fill the pixel to be filled in the first image to generate a predicted frame image displayed between the first real frame image and the second real frame image; wherein the color data of the valid pixels in the first image are retained.

2. The method according to claim 1, characterized in that The interpolation condition includes that the motion amplitude between two consecutive real frame images is less than an amplitude threshold and there is no semi-transparent object in the real frame image.

3. The method according to claim 2, characterized in that After acquiring the first real frame image and the second real frame image, the method further includes: Determining whether there is a semi-transparent object in the first real frame image; In response to the absence of a semi-transparent object in the first real frame image, calculating a first motion amplitude between the first real frame image and the second real frame image; determining whether the first motion amplitude is less than an amplitude threshold; In response to the first motion amplitude being less than an amplitude threshold, determining that an interpolation condition is satisfied between the first real frame image and the second real frame image; In response to the presence of a semi-transparent object in the first real frame image and / or the first motion amplitude being greater than or equal to an amplitude threshold, it is determined that an interpolation condition is not satisfied between the first real frame image and the second real frame image.

4. The method according to any one of claims 1 to 3, characterized in that: The step of reading the color data of the first real frame image from the memory and using the color data to fill the pixels to be filled in the first image with colors includes: In the first image, determining at least one candidate reference pixel for the pixel to be filled; Determining a target reference pixel among the at least one candidate reference pixel according to the object category to which each candidate reference pixel belongs; According to the relationship between the target reference pixel and the first real frame image, the color data of the target reference pixel is determined by reading the memory; The pixel to be filled is filled with color based on the color data of the target reference pixel.

5. The method according to claim 4, characterized in that Determining, in the first image, candidate reference pixels for the pixels to be filled, comprising: Taking the position of the pixel to be filled as the starting point, randomly move to the first position in the first image; If the pixel at the first position is not the pixel to be filled, determining the pixel at the first position as a candidate reference pixel of the pixel to be filled; If the pixel at the first position is a pixel to be filled, starting from the first position, continue to move randomly in the first image until moving to a second position that is not a pixel to be filled, and determine the second position as a candidate reference pixel for the pixel to be filled.

6. The method according to claim 4, characterized in that The color filling of the pixel to be filled based on the color data of the target reference pixel includes: Determining effective color data of the target reference pixel according to a weight corresponding to the target reference pixel and color data of the target reference pixel; Accumulating the valid color data to the existing color data of the pixel to be filled until the number of color accumulations of the pixel to be filled is equal to a first threshold, thereby obtaining the current color data of the pixel to be filled; Dividing the current color data by the first threshold value to obtain first color data of the pixel to be filled; The first color data is written into the position of the pixel to be filled in the first image.

7. The method according to any one of claims 1 to 6, characterized in that: The step of processing the first real frame image to obtain a first image and a mask image of the first image includes: According to the object category to which each pixel in the first real frame image belongs, using a motion vector to perform two image warping operations on the first real frame image to generate a first image; A mask image of the first image is generated according to whether color data exists for each pixel in the first image.

8. The method according to any one of claims 1 to 6, characterized in that: The step of processing the first real frame image to obtain a first image and a mask image of the first image includes: generating a template cache of the first real frame image according to the category of the object to which each pixel in the first real frame image belongs; Querying the template cache to determine the object category to which each pixel in the mask texture image belongs, the mask texture image being an image whose initial values ​​are all first values, the first value being used to mark pixels to be filled in the first image; According to the object category to which each pixel belongs, the first real frame image and the mask texture map are respectively subjected to two image distortion operations using a motion vector to generate a first image and a mask image of the first image.

9. The method according to claim 8, characterized in that The method of performing two image warping operations on the first real frame image and the mask texture map respectively using a motion vector according to the object category to which each pixel belongs, to generate a first image and a mask image of the first image, comprises: Using a multi-rendering target MRT technology, a first image warping operation corresponding to a motion vector is simultaneously performed on the first real frame image and the pixels in the mask texture map belonging to the first object category to generate a warped first real frame image and a warped mask texture map; Performing a second image warping operation corresponding to the motion vector on the pixels belonging to the second object category in the warped first real frame image and the warped mask texture image at the same time, to generate a first image and a mask image of the first image; The first object category is a static object, and the second object category is a dynamic object, or the first object category is a dynamic object, and the second object category is a static object.

10. The method according to any one of claims 1 to 9, characterized in that: The step of sampling the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled includes: Sampling the mask data of each pixel in the mask image pixel by pixel; In the mask image, if the mask data of the sampled pixel is equal to a first value, determining that the pixel at the corresponding position in the first image is a pixel to be filled; If the mask data of the sampled pixel is equal to the second value, it is determined that the pixel at the corresponding position in the first image is not a pixel to be filled.

11. The method according to claim 10, characterized in that The mask image is a binary image, the first value is equal to 1, and the second value is equal to 0.

12. A data processing device, characterized in that: include: An acquisition unit, used for acquiring a first real frame image and a second real frame image; a processing unit, configured to process the first real frame image when an interpolation condition is satisfied between the first real frame image and the second real frame image, to obtain a first image and a mask image of the first image, wherein the mask image is used to mark pixels to be filled and valid pixels in the first image; The processing unit is further used to sample the mask data of the mask image pixel by pixel to determine whether each pixel in the first image is a pixel to be filled; A filling unit is used to read the color data of the first real frame image from the memory for any pixel to be filled in the first image, and use the color data to fill the pixel to be filled in the first image, so as to generate a predicted frame image displayed between the first real frame image and the second real frame image; wherein the color data of the valid pixels in the first image are retained.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the data processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the computer executes the data processing method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables an electronic device to execute a data processing method as claimed in any one of claims 1 to 11.

Images