Image processing methods, electronic devices, chip systems, storage media and software products

Abstract

This application provides an image processing method, electronic device, chip system, storage medium, and program product, applied in the field of terminal technology. It helps avoid problems such as jittering of target object shadows and mismatch between target object shadows and environmental textures in game scenes, thus improving user experience. The method includes: generating a third image based on a first image and a second image; wherein the third image includes a target object and a third shadow of the target object, the area occupied by the third shadow includes a first region and a second region, the pixel value of the first region is obtained based on the pixel value of the third region in the first shadow, and the pixel value of the second region is obtained based on the pixel value of the fourth region in the second shadow; the first region and the third region are the same region in the same coordinate system, and the second region and the fourth region are the same region in the same coordinate system.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method, electronic device, chip system, storage medium and program product. Background Technology

[0002] With the increasing performance of electronic devices, they can now meet the performance requirements of various applications. For example, users can experience online games where players can complete tasks solo or in teams, allowing them to play anytime, anywhere. To enhance the user experience, some game scenarios can enable character shadows to increase immersion.

[0003] However, the character shadows seen by users during gameplay may flicker or not match the ground texture in the game scene, affecting the user experience. Summary of the Invention

[0004] This application provides an image processing method, electronic device, chip system, storage medium, and program product, which are applied in the field of terminal technology. They help to avoid problems such as target object shadow jitter and target object shadow mismatch with environmental texture in game scenes, thereby improving user experience.

[0005] In a first aspect, embodiments of this application propose an image processing method applied to an electronic device. The method includes: acquiring a first image and a second image, wherein the first image includes a target object and a first shadow of the target object, and the second image includes the target object and a second shadow of the target object; generating a third image based on the first image and the second image; wherein the third image includes the target object and a third shadow of the target object, the area occupied by the third shadow includes a first region and a second region, the pixel value of the first region is obtained based on the pixel value of the third region in the first shadow, and the pixel value of the second region is obtained based on the pixel value of the fourth region in the second shadow; the first region and the third region are the same region in the same coordinate system, and the second region and the fourth region are the same region in the same coordinate system; and sequentially displaying the first image, the third image, and the second image.

[0006] In this embodiment, the third shadow in the third image occupies a first region and a second region. The pixel value of the first region is obtained based on the pixel value of the third region in the first shadow, and the pixel value of the second region is obtained based on the pixel value of the fourth region in the second shadow. Furthermore, the first and third regions are the same region in the same coordinate system, and the second and fourth regions are also the same region in the same coordinate system. In other words, the shadow color of the third shadow in the third image is obtained from the third region in the first image corresponding to the first region, and the fourth region in the second image corresponding to the second region. Since the first and third regions are the same region in the same coordinate system, and the second and fourth regions are the same region in the same coordinate system, their textures also correspond. This approach helps avoid the problem of mismatch between the texture in the third shadow and the environmental texture in the third image. When the electronic device displays the first, third, and second images sequentially, the user will not perceive shadow jitter or a mismatch between the shadow and the environmental texture, providing a better user experience.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, generating a third image based on the first image and the second image includes: obtaining a first motion vector, the first motion vector representing the displacement between the position of the same target in the first image and its position in the third image; performing motion compensation on the first image based on the first motion vector to generate a fourth image, the fourth image including pixels whose pixel values ​​are to be modified, the pixels whose pixel values ​​are to be modified including pixels in the first region and pixels in the second region; modifying the pixel values ​​of the pixels whose pixel values ​​are to be modified in the fourth image based on the pixel values ​​of each pixel in the first image and the pixel values ​​of each pixel in the second image, to generate the third image.

[0008] It should be understood that the difference between the fourth image and the third image lies in the fact that the fourth image contains pixels whose pixel values ​​need to be modified. These pixels include the pixels in the first region and the pixels in the second region occupied by the third shadow. In game applications, the fourth image could be, for example,... Figure 8 The predicted frame shown does not include the shadows of the figures.

[0009] In this embodiment, during the generation of the third image, a fourth image is generated first. The fourth image includes pixels whose pixel values ​​need to be modified, including pixels from the first region and pixels from the second region. In other words, the difference between the fourth and third images is that the fourth image contains pixels whose pixel values ​​need to be modified; these pixels include those from the first and second regions occupied by the third shadow. This method of generating the fourth image first helps avoid the subsequent pixel value filling process affecting the remaining pixels besides those whose pixel values ​​need modification, thus ensuring the quality of the third image.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a target object shadow mask, the target object shadow mask being used to mark the position of the shadow of the target object in the image; performing motion compensation on the first image based on the first motion vector to generate a fourth image, including: performing motion compensation on the position of each target pixel in the first image based on the first motion vector to obtain the position of each pixel in the fourth image corresponding to each target pixel in the first image; setting the pixel value of each pixel in the fourth image after motion compensation based on the pixel value of each target pixel in the first image, the relationship between the position of each target pixel in the first image and the position marked by the target object shadow mask, and the relationship between the position of each target pixel in the fourth image after motion compensation and the position marked in the target object shadow mask, and generating the fourth image.

[0011] It should be understood that the target object shadow mask is a matrix or image of the same size as the first image, second image, and third image. The target object shadow mask marks the position of the target object's shadow in the image; that is, the position of the first shadow in the first image, the position of the second shadow in the second image, and the position of the third shadow in the third image can be the same. For example, the target object shadow mask could be... Figure 6 As shown.

[0012] Since the position of the third shadow in the third image can be consistent with the position of the target shadow in the target object shadow mask, that is, the positions of the first shadow in the first image, the second shadow in the second image, and the third shadow in the third image can be the same, when the electronic device displays the first image, the third image, and the second image in sequence, the user may not feel shadow jitter, which is beneficial to improving the image display quality of the electronic device and improving the user experience.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, setting the pixel value of each pixel corresponding to the motion-compensated target pixel in the fourth image includes: if the position of the target pixel in the first image belongs to the position marked in the target object shadow mask, or if the position of the target pixel in the first image does not belong to the position marked in the target object shadow mask, but the position of the target pixel in the fourth image after motion compensation belongs to the position marked in the target object shadow mask, then setting the pixel value of the pixel corresponding to the motion-compensated target pixel in the fourth image to a first preset value, the first preset value indicating that the pixel value needs to be modified; if the position of the target pixel in the first image does not belong to the position marked in the target object shadow mask, and the position of the target pixel in the fourth image after motion compensation also does not belong to the position marked in the target object shadow mask, then setting the pixel value of the pixel corresponding to the motion-compensated target pixel in the fourth image to the pixel value of the target pixel.

[0014] In this embodiment, the electronic device can traverse each target pixel in the first image, perform motion compensation based on the first motion vector to obtain the fourth image, and set the pixel values ​​of each target pixel in the fourth image after motion compensation based on the pixel values ​​of each target pixel in the first image, the relationship between the position of each target pixel in the first image and the position of the target object shadow mask mark, and the relationship between the position of each target pixel in the fourth image after motion compensation and the position of the target object shadow mask mark, so as to obtain the fourth image including pixels with pixel values ​​to be modified, which is beneficial for obtaining the third shadow in the third image based on the first shadow in the first image and the second shadow in the second image.

[0015] It should be understood that the fourth image can be obtained either by obtaining the pixel value of the first preset value during the motion compensation process, or by setting the pixel value of each pixel of the first image to the first preset value after writing the pixel value of each pixel of the first image into the image. The specific method by which this application achieves this purpose is not limited.

[0016] In one possible implementation, setting the pixel value of each pixel corresponding to the target pixel in the fourth image after motion compensation includes: setting the pixel value of the pixel corresponding to the target pixel in the fourth image after motion compensation to be the pixel value of the target pixel; setting the pixel value of the pixel at each position of the target pixel corresponding to the target pixel in the fourth image that belongs to the position marked in the target object shadow mask to a first preset value, the first preset value being used to indicate that the pixel value needs to be modified; and setting the pixel value of the pixel corresponding to the target pixel in the fourth image after motion compensation to the first preset value when the position of the target pixel in the first image belongs to the position marked in the target object shadow mask.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a second motion vector, the second motion vector representing the displacement between the position of the same target in the second image and its position in the third image; modifying the pixel value of a pixel in the fourth image to be modified based on the pixel values ​​of each pixel in the first image and the pixel values ​​of each pixel in the second image includes: obtaining the pixel in the fourth image to be modified; determining the position of the corresponding pixel in the first image based on the first motion vector; and determining the position of the pixel to be modified based on the second motion vector. The position of the pixel whose pixel value is to be modified in the second image; if the position of the pixel whose pixel value is to be modified in the fourth image is a position marked in the target object shadow mask: if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified is a position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the first image, the pixel whose pixel value is to be modified is included in the first region, and the pixel corresponding to the pixel whose pixel value is to be modified in the first image includes... In the third region; if the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified belongs to the position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the second image. The pixel whose pixel value is to be modified is included in the second region, and the pixel corresponding to the pixel whose pixel value is to be modified in the second image is included in the fourth region; if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified does not belong to the target object shadow mask. If the position marked in the first image and the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified is not also the position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to any one of the following: the pixel value of the pixel in the first image whose corresponding pixel value is to be modified belongs to the position marked in the target object shadow mask within a preset radius, the pixel value of the pixel in the second image whose corresponding pixel value is to be modified belongs to the position marked in the target object shadow mask within a preset radius, or a second preset value.

[0018] In this way, the pixels whose pixel values ​​in the fourth image are to be modified, whose positions in the fourth image belong to the positions marked in the shadow mask of the target object, have pixel values ​​that come from the first shadow of the first image and / or the second shadow of the second image. The pixel values ​​of the pixels whose pixel values ​​in the fourth image are to be modified can be understood as being copied from the corresponding pixels in the first shadow of the first image and / or the second shadow of the second image. There will be no texture mismatch, which is beneficial to improving the user experience.

[0019] Furthermore, for the pixels in the fourth image whose pixel values ​​need to be modified, those pixels whose positions in the fourth image do not belong to the positions marked in the target object shadow mask have a very small proportion in the entire image and have little impact on the overall image. Therefore, the pixel values ​​of the pixels in this area whose pixel values ​​need to be modified can be the pixel values ​​of the pixels within a preset radius of the position of the pixel corresponding to the pixel in the first image that belong to the positions marked in the target object shadow mask (e.g., the pixel value of the pixel closest to the pixel corresponding to the pixel in the first image that belongs to the positions marked in the target object shadow mask), the pixel values ​​of the pixels within a preset radius of the position of the pixel corresponding to the pixel in the second image that belong to the positions marked in the target object shadow mask, or any of the second preset values, in order to optimize the display effect of the third shadow in the third image.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, if the position of the pixel whose pixel value is to be modified in the fourth image does not belong to the position marked in the target object shadow mask: if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the first image; if the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the second image.

[0021] This facilitates the modification of pixel values ​​in the fourth image that are not located at positions marked in the target object's shadow mask, thus improving the fourth image and obtaining the third image.

[0022] Optionally, if the position of the pixel whose pixel value is to be modified in the fourth image does not belong to the position marked in the target object shadow mask: if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified belongs to the position marked in the target object shadow mask, and the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified belongs to the position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to any one of the following: the pixel value of a pixel within a preset radius of the position of the pixel corresponding to the pixel in the first image that does not belong to the position marked in the target object shadow mask, the pixel value of a pixel within a preset radius of the position of the pixel corresponding to the pixel in the second image that does not belong to the position marked in the target object shadow mask, or a third preset value.

[0023] The third preset value can be any value that allows a pixel to display a non-shadow color. In game application scenarios, the third preset value can be the average pixel value of the non-shadow area stored in the history of the electronic device, or any pixel value of the non-shadow area stored in the history. This application does not limit this.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, obtaining the pixel points whose pixel values ​​are to be modified in the fourth image includes: traversing each pixel point in the fourth image and obtaining the pixel points whose pixel values ​​are to be modified as the pixel points whose pixel values ​​are to be modified.

[0025] In this way, the traversal method is beneficial to obtain all the pixels in the fourth image whose pixel values ​​need to be modified, which is beneficial to the fact that the pixel values ​​of these pixels can be modified, which is beneficial to the reduction of the possibility of missing pixel values ​​in the fourth image, and which is beneficial to the improvement of the completeness of the final third image.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, obtaining the target object shadow mask includes: obtaining a fifth image, which is an image generated during the first image generation process when the first shadow is started; obtaining a sixth image, which is an image generated during the first image generation process when the first shadow is completed; subtracting the pixel values ​​of each pixel in the sixth image and the fifth image respectively, and marking the positions of pixels whose pixel difference after subtraction is not zero, and / or marking the positions of pixels whose pixel difference after subtraction is zero, to obtain the target object shadow mask.

[0027] This will not affect the process of generating the first image (or the second image), and will facilitate the normal output of the first image (or the second image).

[0028] In conjunction with the first aspect, in some implementations of the first aspect, the electronic device includes a game application that has a shadow function enabled, and the first image and the second image are images drawn based on drawing instructions from the game application.

[0029] Secondly, embodiments of this application provide an image processing apparatus, which may be an electronic device, a chip, or a chip system within an electronic device. The image processing apparatus may include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit may be a display screen. The display unit is used to perform display steps to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is an electronic device, the processing unit may be a processor. The image processing apparatus may further include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or a chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0030] Thirdly, embodiments of this application provide an electronic device including one or more processors and a memory, the memory being coupled to one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and one or more processors calling the computer instructions to cause the electronic device to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0031] Fourthly, embodiments of this application provide a computer-readable storage medium, which includes a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0032] Fifthly, embodiments of this application provide a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0033] Sixthly, this application provides a chip or chip system including one or more processors and a communication interface. The communication interface and the one or more processors are interconnected via a circuit. The one or more processors are used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0034] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0035] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0036] Figure 1 A schematic diagram illustrating the order in which image frames are displayed on an electronic device according to an embodiment of this application;

[0037] Figure 2 A schematic diagram of a problem scenario provided in an embodiment of this application;

[0038] Figure 3 This is a schematic diagram illustrating another problem scenario provided in an embodiment of this application;

[0039] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0040] Figure 5 A schematic flowchart illustrating an image processing method provided in an embodiment of this application;

[0041] Figure 6 A schematic diagram illustrating an image processing procedure provided in an embodiment of this application;

[0042] Figure 7 A schematic flowchart illustrating an image processing method provided in an embodiment of this application;

[0043] Figure 8A schematic diagram of a prediction frame that does not include shadows of people, provided as an embodiment of this application;

[0044] Figure 9 A schematic diagram of the partitioning of region 1 in a prediction frame that does not contain human shadows, provided for an embodiment of this application;

[0045] Figure 10 A schematic flowchart illustrating an image processing method provided in an embodiment of this application;

[0046] Figure 11 A schematic diagram illustrating the comparison between a real frame and a predicted frame, provided as an embodiment of this application;

[0047] Figure 12 A schematic flowchart illustrating an image processing method provided in an embodiment of this application;

[0048] Figure 13 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0049] Figure 14 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0050] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0051] 1. Terminology

[0052] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0053] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0054] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0055] 2. Electronic equipment

[0056] The electronic devices in this application embodiment may include handheld devices, vehicle-mounted devices, etc., with image display functions. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, wearable devices, electronic devices in 5G networks, or future evolution of public land mobile communication networks. The embodiments of this application do not limit the scope of electronic devices in a network (PLMN).

[0057] By way of example and not limitation, in this embodiment, the electronic device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0058] Furthermore, in this application embodiment, the electronic device can also be an electronic device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0059] The electronic device in this application embodiment may also be referred to as: electronic device, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0060] In this embodiment, the electronic device includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and main memory. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0061] When electronic devices run online games, they typically need to render a large number of complex game interfaces. The rendering process usually consumes a lot of central processing unit (CPU) resources, which can easily lead to a heavy load on the electronic device, affecting its battery life. It may also cause other processes running on the electronic device at the same time as the game application to lag. Therefore, in related technologies, in order to reduce the rendering overhead of electronic devices when running online games, the image frames sent for display include a portion of the frames output after the actual rendering and other image processing processes (hereinafter referred to as real frames) and a portion of the predicted frames obtained based on the real frames. This is to ensure that the number of frames actually displayed by the electronic device is reduced while reducing the number of actual rendered game interface frames, so as to ensure the game's image quality and smoothness.

[0062] In some implementations, electronic devices use Figure 1 The output image frames are shown in the order shown. Figure 1 As shown, the electronic device can obtain the predicted frame 12 based on the real frame 1 and the real frame 2, and can obtain the predicted frame 23 based on the real frame 2 and the real frame 3, and then output these frames in the order of real frame 1, predicted frame 12, real frame 2, predicted frame 23, and real frame 3.

[0063] In some scenarios, character shadows can be present in real-world frames of a game scene to provide players with a more realistic and immersive experience. If character shadows are present in real-world frames, then character shadows should also be present in predicted frames. Character shadows are essentially the shadows of characters. To ensure a good gaming experience, the main character (which can be understood as the player's character) is always positioned within a designated area of ​​the game screen (e.g., the center of the screen). In a scenario where the main character is running, the surrounding environment can be used to create the illusion of the main character running forward. In related technologies, character shadows are typically drawn on the surrounding environment (e.g., the ground), while the main character is drawn independently. When generating prediction frames, the character shadows in the prediction frames are estimated using motion estimation of the environment in the preceding and following real frames. This means that in a scenario where the main character is running forward, the character shadow may move backward with the surrounding environment. Since the main character in the prediction frames is usually drawn within a designated area of ​​the screen (meaning the user's view of the main character's position in the screen doesn't change), the relative distance between the main character and their shadow in the prediction frame is greater than the relative distance in the real frame. Furthermore, because the user's view consists of a real frame followed by a prediction frame, and then another real frame, the difference in the relative distance between the main character and their shadow in the real and prediction frames... Figure 2 As shown in the diagram. Figure 2In the diagram, real frames A and B are two adjacent real frames, while predicted frame AB is a frame interpolated between real frames A and B. The numbers 1, 2, 3, and 4 in each frame can be considered as any static object in the environment. It can be seen that the relative distance between the main character and their shadow in predicted frame AB is larger than in real frames A and B. When these frames are played consecutively, the user may see the character's shadow jumping or jittering, resulting in a poor user experience.

[0064] To address the aforementioned issues, related technologies employ a method where, during the generation of a prediction frame, the shadow of the person in the preceding real frame is directly extracted without altering its position within the overall image. This shadow is then used directly as the shadow in the prediction frame. In other words, the shadow in the prediction frame can be predicted without motion compensation; instead, it directly utilizes the shadow from the preceding real frame. Consequently, the relative distance between the main character and their shadow in the prediction frame remains unchanged compared to the relative distance between the main character and their shadow in the real frame, which to some extent resolves the issue of the shadow jumping or jittering.

[0065] However, since the character's shadow is drawn within the surrounding environment, it can be understood as being integrated with the environment. In some implementations, the character's shadow is not completely black; it can have a certain degree of transparency, allowing the user to see the environmental texture through the character's shadow. For example... Figure 3 As shown, when extracting the character shadow 1 from the previous real frame A of the prediction frame, it is impossible to separate the shadow color from the surrounding environment. When the extracted character shadow 1 is then used on the prediction frame AB1, due to the movement of the surrounding environment, the environmental texture in the extracted character shadow 1 is lost. Figure 3 The mesh in the image (which can be understood as the environment texture, such as floor seams) does not match the environment texture in the prediction frame AB1, resulting in an appearance as shown below. Figure 3 The effect shown in the predicted frame AB1 gives users an unrealistic feeling and affects the user experience.

[0066] In view of this, embodiments of this application provide an image processing method, electronic device, chip system, storage medium, and program product. The color of the shadow of a person in the predicted frame is obtained from the shadow of a person in the real frame. Furthermore, the region in the real frame from which the color of the shadow of the person in the predicted frame is obtained belongs to the same region as the region in the predicted frame where the shadow color is filled in, in the entire game map scene coordinate system. Therefore, the texture of the color of the shadow of the person in the predicted frame obtained in this way is a texture that matches the surrounding environment.

[0067] It should be understood that the first image and the second image can be real frames drawn based on the drawing instructions of the game application. The first image and the second image can also be understood as two images obtained when the target object moves to different positions on the game scene map, where the target object is in a specified position on the game screen in both images. For example, the first image could be... Figure 11 The real frame A shown in the image can have the first shadow as, for example, the shadow of a person in real frame A, and the second image as, for example, the shadow of a person in real frame A. Figure 11 The second shadow, as shown in the real frame B, could be, for example, the shadow of a person in real frame B. Furthermore, the third image could be predicted based on the first and second images, for example, it could be... Figure 11 The third shadow in the prediction frame AB 2 shown in the figure could be, for example, the shadow of the figure 4 in the prediction frame AB 2.

[0068] It should also be understood that the color here refers to the visual attributes of the image from the user's perspective, which can be understood as the pixel values ​​of each pixel in the image.

[0069] It is worth noting that the image processing method provided in this application embodiment can be applied to any scene where frame interpolation can be performed using predicted frames and the predicted frames include shadows of moving objects. The description of the embodiments in this application is only used as an example of the application of predicted frames to game applications and should not constitute a specific limitation on this application.

[0070] The software structure of the electronic device involved in the embodiments of this application will be introduced first below.

[0071] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software structure of an electronic device.

[0072] Figure 4 An exemplary schematic diagram of the software architecture of an electronic device is shown.

[0073] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system layer, and the kernel layer.

[0074] The application layer can include a series of application packages. For example... Figure 4 As shown, an application package may include, for example, game applications or applications supported by any other electronic device.

[0075] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 4 As shown, the application framework layer can include at least a window manager, a view system, etc.

[0076] The window manager is used to manage windowed applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture the screen, among other things. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0077] The system library layer can include multiple functional modules, such as a graphics processing library and a prediction frame generation module.

[0078] The graphics processing library may include, for example, a 3D graphics processing library (e.g., OpenGL ES) and a 2D graphics processing library (e.g., SGL). In this embodiment, the graphics processing library is used to implement graphics drawing, image rendering, compositing, and layer processing. For example, it can render real frames at runtime, such as real frame A and real frame B described below.

[0079] The prediction frame generation module may include a scene recognition module, a motion estimation module, a shadow filling module, and an image completion module. Specifically, the scene recognition module can be used to obtain a character shadow mask (also called a character shadow veil or character shadow masking image) during the rendering of a real frame in the graphics processing library; the motion estimation module can be used to calculate motion vectors based on real frame A and real frame B; the motion compensation module can be used to perform motion compensation based on the motion vectors and real frame B (or real frame A) and generate a prediction frame that does not contain character shadows, where there are regions with pixel values ​​to be modified; the shadow filling module is used to fill the regions with pixel values ​​to be modified in the prediction frame that does not contain character shadows to generate a prediction frame containing character shadows; and the image completion module completes the missing pixel regions in the prediction frame.

[0080] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, graphics processor driver, etc.

[0081] It should be understood that the software architecture of an electronic device may include more or fewer layers, and the names of each layer and the software modules are merely exemplary and do not constitute a limitation on the electronic device involved in this application.

[0082] For ease of understanding, the following describes in detail an image processing method provided in this application embodiment, taking the generation of predicted frames in a game scene as an example. Figure 5 An exemplary embodiment of this application illustrates a method 500 for generating a predicted frame AB2 based on the aforementioned real frame A and real frame B. This method 500 can be applied to electronic devices, and the software structure of the electronic device can be as follows: Figure 4 As shown, the method 500 includes the following steps:

[0083] S501. Based on the rendering process of real frames, obtain a character shadow mask image, which includes the area marked with the character shadow in the image.

[0084] It should be understood that a person's shadow mask image can be a matrix or image of the same size as the real frame, used to indicate or identify pixels or regions of person's shadow in the real frame. The pixel values ​​of each pixel in the person's shadow mask image can be binary (e.g., containing only 0, 1, or other arbitrary values), or they can be multi-channel, with each channel potentially representing different types of information or targets. The changes between two consecutive real frame images are often relatively small. The person's shadow region marked in the person's shadow mask image can be the same region in real frame A, real frame B, and predicted frame AB2.

[0085] Optionally, a possible rendering process for realistic frames could be: 1) First, render the environment: The environment is the background and foundation of the game, providing the overall atmosphere and visual effects. When rendering the environment, the influence of factors such as lighting, weather, and time on the scene needs to be considered; 2) Then, render the characters: After the environment is rendered, the character models are placed in the scene according to the game logic and player actions, and skeletal animation and lighting processing are performed. The rendering of the character models needs to be depth-tested and composited with the environment to ensure the correct interaction between the characters and the environment; 3) Finally, render the character shadows: After the characters are rendered, shadows are generated for the characters according to the lighting conditions and the character's position, and then composited with the scene and the characters. The rendering of the shadows needs to ensure that the position, shape, and intensity of the shadows match the lighting conditions to enhance the three-dimensionality and realism of the image.

[0086] In this embodiment of the application, the generation of the real frame can be implemented by the graphics processing library in the electronic device in response to the drawing instructions from the application. The scene recognition module of the predictive frame generation module in the electronic device can intercept and recognize the drawing instructions. If the current drawing instructions are used to indicate the drawing of the character's shadow, shadow 1 before the drawing of the character's shadow is obtained. If the drawing of the character's shadow is completed, shadow 2 after the drawing of the character's shadow is obtained.

[0087] In some implementations, electronic devices can subtract the pixel values ​​of pixels with the same pixel coordinates in shadow 2 from those in shadow 1, marking the positions of pixels with a difference of zero as non-human shadow areas, and / or marking the positions of pixels with a difference of non-zero as human shadow areas. This process can be as follows: Figure 6 As shown. Figure 6 The “drawing other factors” shown in the document may be an optional step. “Drawing other factors” may include drawing the environment and / or drawing people. This application does not limit the specific content of other factors.

[0088] For example, the real frame and the predicted frame are images of the same size. The pixel coordinates can be formed by taking any vertex of the image frame as the origin and taking the two perpendicular edges of the image frame extending from that vertex as the two coordinate axes. The pixel coordinates can be the coordinates in this pixel coordinate system.

[0089] Optionally, the drawing command for the character's shadow can be, for example, glclearcolor(1.0, 1.0, 1.0, 0.0). The electronic device can obtain shadow 1 and shadow 2 by using one or more of the following methods: glBliteFramebuffer, GL_EXT_shader_framebuffer_fetch, switching Framebuffer, etc. This application does not limit this.

[0090] S502, Obtain the motion vector 1 of the predicted frame AB2 relative to the real frame A, and the motion vector 2 of the predicted frame AB2 relative to the real frame B.

[0091] In some implementations, combined Figure 4 The above-mentioned S502 can be executed by the motion estimation module in the prediction frame generation module included in the electronic device. The motion estimation module can first calculate the motion vector between the real frame A and the real frame B based on the pixel value and / or position (which can be represented by pixel coordinates) of each pixel point in the real frame A and the real frame B. Then, based on the interpolation position of the prediction frame AB2 between the real frame A and the real frame B, it can obtain the motion vector 1 of the prediction frame AB2 relative to the real frame A and the motion vector 2 of the prediction frame AB2 relative to the real frame B.

[0092] In other implementations, if the game application already has motion vector 1 and motion vector 2, the motion estimation module can obtain motion vector 1 and / or motion vector 2 directly from the game application without calculation. This application does not limit this.

[0093] Of course, it should also be understood that the descriptions of the entities performing each step in this application are all exemplary and should not constitute a specific limitation on this application.

[0094] S503. Using the real frame A, motion vector 1, and the character shadow mask image, obtain a prediction frame that does not contain the character shadow. The prediction frame that does not contain the character shadow includes the region 1 whose pixel value needs to be modified.

[0095] It should be understood that motion vector 1 can be a vector matrix, which may include the motion vector of each pixel in the real frame A relative to the predicted frame AB 2, but this application does not limit it.

[0096] In some implementations, the motion vector corresponding to the main character in the vector matrix of motion vector 1 can be zero.

[0097] In some implementations, the motion compensation module in an electronic device can use the real frame A as the prediction basis and, based on the motion vector 1 obtained by the motion estimation module, predict the prediction frame that does not contain the shadow of the person to obtain the prediction frame that does not contain the shadow of the person. This process can be called motion compensation. It should be understood that this application only uses the real frame A and motion vector 1 as an example for motion compensation. In other implementations, motion compensation can of course be performed using the real frame B and motion vector 2. The method described here is only an example and does not constitute a specific limitation on this application.

[0098] Alternatively, the electronic device may employ a global motion compensation method or a segmented motion compensation method for motion compensation.

[0099] For example, Figure 7 The process 700 of motion compensation by an electronic device is illustrated. It includes the following steps:

[0100] S5031. For pixel 1 in real frame A, determine whether the position of pixel 1 in real frame A belongs to the shadow area of ​​the character in the character shadow mask image; if yes, execute S5302; if no, execute S5303.

[0101] S5032. Motion compensation is performed on pixel 1 based on the motion vector of pixel 1 included in motion vector 1 to obtain pixel 11 corresponding to pixel 1 in the prediction frame that does not contain the shadow of the person. Pixel 11 is marked as a pixel whose pixel value is to be modified. The pixel value of pixel 1 is not written into pixel 11.

[0102] S5033. Motion compensation is performed on the pixel based on the motion vector of the pixel 1 included in the motion vector 1 to obtain the pixel 11 corresponding to the pixel in the prediction frame that does not contain the shadow of the person.

[0103] S5034. Determine whether the position of pixel 11 in the prediction frame that does not contain the shadow of the person belongs to the shadow area of ​​the person in the shadow mask image. If yes, execute S5035; otherwise, execute S5036.

[0104] S5035, Mark pixel 11 as the pixel whose pixel value is to be modified. Pixel 11 has not been written with the pixel value of pixel 1.

[0105] S5036. Write the pixel value of pixel 1 to pixel 11.

[0106] S5037. Traverse each pixel in the real frame A to obtain a predicted frame that does not contain the shadow of the person. The predicted frame that does not contain the shadow of the person contains a region 1 formed by the pixels whose pixel values ​​are to be modified.

[0107] Figure 8 An example is shown, which is a predicted frame that does not include shadows of people, obtained based on the real frame A.

[0108] It should be understood that the above motion compensation process may result in some pixels, besides region 1, being missing in the predicted frame that does not contain the shadow of the person, for example... Figure 8 The predicted frames shown do not include character shadows and Figure 2 Compared to the real frame A, the predicted frame, which should not contain the shadow of the person, should contain a static object 4 that is not present in the real frame A. Therefore, the electronic device can perform image completion for these areas to improve the accuracy of the obtained predicted frame. Optionally, the electronic device can use the static object 4 present in the real frame B to perform motion compensation for image completion, or perform image completion based on deep learning to generate similar texture regions, etc., and this application does not limit this method.

[0109] Optionally, the image completion module in the electronic device can perform image completion before or after region 1 is filled, and this application does not limit this.

[0110] Figure 9 An illustrative example shows the composition of region 1 in a prediction frame that does not contain a person's shadow. It can be seen that region 1 is formed by the overlap of person's shadow 1 in the ground truth frame A and person's shadow 4 in the prediction frame AB 2, determined by the person's shadow mask map. It should be understood that a prediction frame AB 2 containing person's shadow 4 can be obtained by filling region 1 in the prediction frame that does not contain person's shadow.

[0111] For example, the motion compensation process of the electronic device can also be as follows: For pixel 1 in the real frame A, motion compensation is performed on pixel 1 based on the motion vector of pixel 1 included in motion vector 1 to obtain pixel 11 corresponding to pixel 1 in the prediction frame that does not contain human shadows, and the pixel value of pixel 1 is written to pixel 11; For pixel 11, if the position of pixel 11 in the prediction frame that does not contain human shadows belongs to the human shadow area in the human shadow mask image, then pixel 11 is set as the pixel with the pixel value to be modified; For pixel 11, if the position of pixel 1 corresponding to pixel 11 in the real frame A belongs to the human shadow area in the human shadow mask image, then pixel 11 is set as the pixel with the pixel value to be modified; Traversing each pixel in the prediction frame that does not contain human shadows, a prediction frame that does not contain human shadows and includes region 1 formed by the pixel with the pixel value to be modified is obtained. This application does not limit the specific method by which the electronic device obtains the prediction frame that does not contain human shadows.

[0112] S504. Based on the predicted frame excluding the shadow of the person and motion vector 1 and motion vector 2, obtain the pixel value of the corresponding pixel in region 1 from the real frame A or the real frame B and fill it into region 1 to obtain the predicted frame AB 2.

[0113] For example, Figure 10 A possible method 1000 for filling region 1 in a predicted frame that does not include the shadow of a person is shown. This method is applied to an electronic device and can be specifically performed by a shadow filling module in the electronic device. The method 1000 includes the following steps:

[0114] S5041. For the i-th pixel 2 in the prediction frame that does not include the shadow of the person, determine whether the pixel 2 is a pixel in region 1. If yes, execute S5042. If no, return and continue to judge the i-th (i+1)-th pixel until all pixels in the prediction frame that does not include the shadow of the person have been traversed, where i is an integer greater than zero.

[0115] In one possible implementation, in method 700 described above, the method for marking the pixel whose pixel value is to be modified can be to set a marker value α in any of its pixel channels (Red, Green, Blue, and Alpha, for example, the Alpha channel), and / or to write a motion vector (or pixel coordinates, or any value that can distinguish it from non-pixel values ​​to be modified) in any of the RGBA channels. Therefore, exemplarily, here, the electronic device can determine whether pixel 2 is a pixel in region 1 by checking whether a marker value α exists in the RGBA channel of pixel 2.

[0116] S5042. Determine whether pixel 2 belongs to the shadow area of ​​the person in the shadow mask image. If yes, execute S5043; otherwise, execute S50410.

[0117] It should be understood that the pixel values ​​of the shadow area of ​​the person in the person shadow mask image can be marked as preset value 1, for example, 1. The pixel values ​​of other non-shadow areas can be marked as preset value 2, for example, 0. In some implementations, the method to determine whether pixel 2 belongs to the shadow area of ​​the person in the person shadow mask image can be to check whether the preset value of the pixel with the same pixel coordinates as pixel 2 in the person shadow mask image is preset value 1. If it is, then pixel 2 can be considered to belong to the shadow area of ​​the person in the person shadow mask image; if not, then pixel 2 can be considered not to belong to the shadow area of ​​the person in the person shadow mask image.

[0118] It should be understood that the method for determining whether pixel 21 and pixel 22 belong to the shadow area of ​​the person in the shadow mask image is similar to the method for determining whether pixel 2 belongs to the shadow area of ​​the person in the shadow mask image, and will not be described again later.

[0119] S5043. Based on the motion vector of pixel 2 included in motion vector 1, determine the pixel 21 corresponding to pixel 2 in real frame A.

[0120] S5044. Determine whether pixel 21 belongs to the shadow area of ​​the person in the shadow mask image. If yes, execute S5045; otherwise, execute S5046.

[0121] S5045, Obtain the pixel value of pixel 21 and fill it into pixel 2 in the prediction frame that does not include the shadow of the person.

[0122] S5046. Based on the motion vector corresponding to pixel 2 included in motion vector 2, determine pixel 22 corresponding to pixel 2 in real frame B.

[0123] S5047. Determine whether pixel 22 belongs to the shadow area of ​​the person in the shadow mask image. If yes, execute S5048; otherwise, execute S5049.

[0124] S5048, Obtain the pixel value of pixel 22 and fill it into pixel 2 in the prediction frame that does not include the shadow of the person.

[0125] S5049. Fill in the pixel value of the pixel that belongs to the shadow area of ​​the character near pixel 21 at pixel 2, or fill in the pixel value of the pixel that belongs to the shadow area of ​​the character near pixel 22.

[0126] S50410. Based on the motion vector of pixel 2 included in motion vector 1, determine pixel 21 corresponding to pixel 2 in real frame A.

[0127] S50411. Determine whether pixel 21 belongs to the shadow area of ​​the person in the shadow mask image. If not, execute S50512. If yes, execute S50413.

[0128] S50412, Obtain the pixel value of pixel 21 and fill it into pixel 2 in the prediction frame that does not include the shadow of the person.

[0129] S50413. Based on the motion vector corresponding to pixel 2 included in motion vector 2, determine pixel 22 corresponding to pixel 2 in real frame B.

[0130] S50414. Determine whether pixel 22 belongs to the shadow area of ​​the person in the shadow mask image. If yes, execute S50416; otherwise, execute S50415.

[0131] S50415, Obtain the pixel value of pixel 22 and fill it into pixel 2 in the prediction frame that does not include the shadow of the person.

[0132] S50416. Fill in pixel value of a pixel that is not in the shadow area of ​​the person near pixel 21 at pixel 2, or fill in pixel value of a pixel that is not in the shadow area of ​​the person near pixel 22.

[0133] It should be understood that the area occupied by the shadow of the person in the real frame A and the real frame B may be slightly different due to the change of the person's posture. Thus, there may be a situation where pixel 2 in the prediction frame that does not include the shadow of the person is both the corresponding pixel 21 in the real frame A and the corresponding pixel 22 in the real frame B, which belong to the shadow area of ​​the person. In this case, the electronic device can execute the above S50416 to obtain the pixel value of pixel 2.

[0134] For ease of understanding, Figure 11 The example illustrates the positional relationship between the shadow of a person in real frame A (1), the shadow of a person in real frame B (3), and region 1 in a predicted frame that does not contain a shadow of a person.

[0135] from Figure 11 The predicted frames shown, excluding human shadows, indicate that region 1 includes regions ①, ②, ③, ④, ⑤, and ⑥ as defined in the diagram. Among these, combining... Figure 9As can be seen, the character shadow 4 in the predicted frame AB 2 is composed of region ①, region ②, region ③, region ⑤, and region ⑥. This combination of regions is also the region corresponding to the character shadow region in the character shadow mask image in the predicted frame that does not contain the character shadow. It is the region to be filled with shadow in this embodiment. The electronic device can find the corresponding shadow region from the real frame A and the real frame B, and fill in the corresponding pixel value of the shadow region. Region ④ does not belong to the character shadow region in the character shadow mask image, so it needs to be filled with a non-shadow region. The electronic device can find the corresponding non-shadow region from the real frame A and the real frame B, and fill in the corresponding pixel value of the non-shadow region.

[0136] Combination Figure 11 As can be seen from the above method 1000, region ① in the predicted frame that does not contain the shadow of the person is not a shadow region in the real frame A, but it is a shadow region in the real frame B. The electronic device can use the region in the real frame B that has the same absolute position as region ① to fill the pixels in region ①. For example, when the electronic device traverses the pixels in region ①, it can execute S5041~S5044, S5046~S5048 to use the region in the real frame B that has the same absolute position as region ① to fill the pixels in region ①.

[0137] Region ② in the predicted frame, which does not contain the shadow of the person, is a shadow region in both real frame A and real frame B. Therefore, the electronic device can use the region in real frame A or real frame B that has the same absolute position as region ② to fill the pixels in region ②. For example, when the electronic device traverses the pixels in region ②, it can execute S5041 to S5045 to use the region in real frame A that has the same absolute position as region ② to fill the pixels in region ②.

[0138] Region ③ in the predicted frame, which does not contain the shadow of the person, is a shadow region in the real frame A, but not in the real frame B. Therefore, the electronic device can use the region in the real frame A that has the same absolute position as region ③ to fill the pixels in region ③. For example, when the electronic device traverses the pixels in region ③, it can execute S5041 to S5045 to use the region in the real frame A that has the same absolute position as region ③ to fill the pixels in region ③.

[0139] Regions ⑤ and ⑥ in the predicted frame that do not contain the shadow of the person are not shadow regions in either the real frame A or the real frame B. Regions ⑤ and ⑥ are generated due to the change between the predicted frame AB2 and the real frames A and B caused by motion. However, in reality, the change between the predicted frame AB2 and the real frames A and B is small. Regions ⑤ and ⑥ are small overall, and their pixel colors have little impact on the shadow regions. Therefore, the electronic device can use the region in the real frame A or the real frame B that is in the same absolute position as regions ⑤ and ⑥ to fill the pixels in regions ⑤ and ⑥, or use the shadow region in the real frame A or the real frame B that is adjacent to regions ⑤ and ⑥ to fill the pixels in regions ⑤ and ⑥. This application does not limit this. For example, when the electronic device traverses the pixels in regions ⑤ and ⑥, it can execute S5041 to S5044, S5046 to S5047, and S5049 to fill the pixels in regions ⑤ and ⑥.

[0140] For region ④ in the predicted frame that does not contain a person's shadow, it does not belong to the person's shadow region in the person's shadow mask image. It belongs to the shadow region in the real frame A, but not in the real frame B. Therefore, the electronic device can use the region in the real frame B that has the same absolute position as region ④ to fill the pixels in region ④. For example, the electronic device can execute the above-described S5041~S5042, or S50410~S50415, or S50410~S50414 and S50416 to fill the pixels in region ④.

[0141] It should be understood that after pixel filling is completed in region 1 of the prediction frame, which does not contain the character's shadow, prediction frame AB 2 is obtained. The character's shadow 4 in prediction frame AB 2 is the result of pixel filling in regions ①, ②, ③, ⑤, and ⑥. Since the character's shadow 4 is filled according to the position corresponding to the character's shadow 4 in the real frame, the phenomenon of mismatch between the character's shadow and the surrounding environment caused by misalignment between the character's shadow 4 and the environmental texture in prediction frame AB 2 can be avoided, which is beneficial to improving the user experience.

[0142] In the above method 1000, after the electronic device executes S5042, regardless of the judgment result, when executing the next step, it first uses real frame A as the judgment basis and then uses real frame B as the judgment basis. However, it should be understood that the judgment order in method 1000 is only exemplary. The electronic device may also use real frame B as the judgment basis and then use real frame A as the judgment basis after executing S5042. This application does not limit this.

[0143] The areas with the same absolute location described here can be understood as the same area in the global map of the game scene, or the same area under the same coordinates.

[0144] Figure 12 This is a schematic flowchart illustrating an image processing method 1200 provided in an embodiment of this application. This method can be applied to electronic devices, and the software structure of the electronic device can be as follows: Figure 4 As shown, the hardware structure can be as follows: Figure 13 As shown, but this application does not limit it.

[0145] The method 1200 includes the following steps:

[0146] S1201. Obtain a first image and a second image. The first image includes the target object and a first shadow of the target object, and the second image includes the target object and a second shadow of the target object.

[0147] S1202. Generate a third image based on the first image and the second image; wherein, the third image includes the target object and the third shadow of the target object, the area occupied by the third shadow includes the first region and the second region, the pixel value of the first region is obtained based on the pixel value of the third region in the first shadow, and the pixel value of the second region is obtained based on the pixel value of the fourth region in the second shadow; the first region and the third region are the same region in the same coordinate system, and the second region and the fourth region are the same region in the same coordinate system.

[0148] S1203, Display the first image, the third image, and the second image in sequence.

[0149] In this embodiment, the third shadow in the third image occupies a first region and a second region. The pixel value of the first region is obtained based on the pixel value of the third region in the first shadow, and the pixel value of the second region is obtained based on the pixel value of the fourth region in the second shadow. Furthermore, the first and third regions are the same region in the same coordinate system, and the second and fourth regions are also the same region in the same coordinate system. In other words, the shadow color of the third shadow in the third image is obtained from the third region in the first image corresponding to the first region, and the fourth region in the second image corresponding to the second region. Since the first and third regions are the same region in the same coordinate system, and the second and fourth regions are the same region in the same coordinate system, their textures also correspond. This approach helps avoid the problem of mismatch between the texture in the third shadow and the environmental texture in the third image. When the electronic device displays the first, third, and second images sequentially, the user will not perceive shadow jitter or a mismatch between the shadow and the environmental texture, providing a better user experience.

[0150] As an optional embodiment, the electronic device includes a game application with a shadow function enabled. The first image and the second image are images drawn based on the drawing instructions of the game application. The first image and the second image can also be understood as two images obtained when a target object moves to different positions in the game scene map. In both images, the target object is at a specified position in the game screen, and the images include the shadow of the target object. For example, the first image may be, for example, a... Figure 11 The real frame A shown in the image can have the first shadow as, for example, the shadow of a person in real frame A, and the second image as, for example, the shadow of a person in real frame A. Figure 11 The second shadow, as shown in the real frame B, could be, for example, the shadow of a person in the real frame B.

[0151] Furthermore, the third image can be predicted based on the first and second images, for example, it could be... Figure 11 The third shadow in the prediction frame AB 2 shown in the figure could be, for example, the shadow of the figure 4 in the prediction frame AB 2.

[0152] In some implementations, the first, second, and third shadows are colored with a certain degree of transparency, and the color of the shadow area includes the color of the environmental texture covered by the shadow. In simpler terms, users can see environmental textures, such as floor stripes or floor seams, in the shadowed area.

[0153] Of course, the first image and the second image can also be two images obtained when the target object in the game application is stationary or moves to the same position in the game scene map. This application does not make specific limitations on this. In both images, the target object is in a specified position in the game screen.

[0154] Optionally, the target object can be an object controlled by the native player in an electronic device game application (such as the main character described above), an object controlled by a non-native player in a multiplayer game scene, or other objects in the environment. This application does not limit this.

[0155] Optionally, the aforementioned coordinate system could be, for example, a unique coordinate system established within the global map of the game scene, and the first region could be, for example, a... Figure 11 The predicted frame AB 2 shown in the figure contains regions ①, ①+region ②, ①+region ⑤, ①+region ⑥, ①+region ⑤+region ⑥, ①+region ②+region ⑤+region ⑥, ①+region ②+region ⑤, or ①+region ②+region ⑥. Correspondingly, the third region could be, for example, region ①+region ②+region ⑥. Figure 11In real frame A, the regions are: region ①, region ① + region ②, region ① + region ⑤, region ① + region ⑥, region ① + region ⑤ + region ⑥, region ① + region ② + region ⑤ + region ⑥, region ① + region ② + region ⑤, or region ① + region ② + region ⑥; the second region can be, for example, region ③, region ③ + region ②, region ③ + region ⑤, region ③ + region ⑥, region ③ + region ⑤ + region ⑥, region ③ + region ② + region ⑤ + region ⑥, region ③ + region ② + region ⑤, or region ③ + region ② + region ⑥. Correspondingly, the fourth region can be, for example, region ①, region ① + region ②, region ⑤, region ⑤ + region ⑥, region ③ + region ② + region ⑥, region ③ + region ② + region ⑤, or region ③ + region ② + region ⑥. Figure 11 The actual frame B contains regions ③, ③+region ②, ③+region ⑤, ③+region ⑥, ③+region ⑤+region ⑥, ③+region ②+region ⑤+region ⑥, ③+region ②+region ⑤, or ③+region ②+region ⑥. This application does not specifically limit these regions in the embodiments.

[0156] As an optional embodiment, one possible implementation of S1202 above includes: obtaining a first motion vector, the first motion vector being used to represent the displacement between the position of the same target in the first image and its position in the third image; performing motion compensation on the first image based on the first motion vector to generate a fourth image, the fourth image including pixels whose pixel values ​​are to be modified, the pixels whose pixel values ​​are to be modified including pixels in the first region and pixels in the second region; modifying the pixel values ​​of the pixels whose pixel values ​​are to be modified in the fourth image based on the pixel values ​​of each pixel in the first image and the pixel values ​​of each pixel in the second image, to generate the third image.

[0157] Optionally, the first motion vector can be the motion vector 1 described in the above embodiments. The meaning of "same target" can be the pixel block or pixel point divided in the first image. The motion vector can be calculated by the block matching algorithm (BMA), pixel recursion method, etc. This application does not make any specific limitations on this.

[0158] In some implementations, the above-mentioned S1202 can be executed by the prediction frame generation module in the electronic device. When the application (e.g., a game application) that outputs the first image and / or the second image has already obtained the first motion vector, the prediction frame generation module can directly obtain the first motion vector from the application that outputs the first image and / or the second image without having to perform calculations based on the first image and the second image. This application does not specifically limit the method of obtaining the motion vector.

[0159] It should be understood that the difference between the fourth image and the third image lies in the fact that the fourth image contains pixels whose pixel values ​​need to be modified. These pixels include the pixels in the first region and the pixels in the second region occupied by the third shadow. In game applications, the fourth image could be, for example,... Figure 8 The predicted frame shown does not include the shadows of the figures.

[0160] It should also be understood that the location described in the embodiments of this application refers to the location of a target in the image where the target is located. This location can be represented by pixel coordinates, the distance between the target and the image border, polar coordinates, etc., and this application does not limit it.

[0161] In this embodiment, during the generation of the third image, a fourth image is generated first. The fourth image includes pixels whose pixel values ​​need to be modified, including pixels from the first region and pixels from the second region. In other words, the difference between the fourth and third images is that the fourth image contains pixels whose pixel values ​​need to be modified; these pixels include those from the first and second regions occupied by the third shadow. This method of generating the fourth image first helps avoid the subsequent pixel value filling process affecting the remaining pixels besides those whose pixel values ​​need modification, thus ensuring the quality of the third image.

[0162] As an optional embodiment, method 1200 further includes: obtaining a target object shadow mask, the target object shadow mask being used to mark the position of the target object's shadow in the image; the above-mentioned motion compensation of the first image based on the first motion vector to generate a fourth image includes: performing motion compensation on the position of each target pixel in the first image based on the first motion vector to obtain the position of each pixel in the fourth image corresponding to each target pixel in the first image; setting the pixel value of each pixel in the fourth image after motion compensation based on the pixel value of each target pixel in the first image, the relationship between the position of each target pixel in the first image and the position marked by the target object shadow mask, and the relationship between the position of each target pixel in the fourth image after motion compensation and the position marked in the target object shadow mask, and generating the fourth image.

[0163] It should be understood that the target object shadow mask is a matrix or image of the same size as the first image, second image, and third image. The target object shadow mask marks the position of the target object's shadow in the image; that is, the position of the first shadow in the first image, the position of the second shadow in the second image, and the position of the third shadow in the third image are the same. For example, the target object shadow mask could be... Figure 6 As shown.

[0164] In one possible implementation, method 1200 further includes: obtaining a second motion vector, which represents the displacement between the position of the same target in the second image and its position in the third image; the method for generating the fourth image may also be: based on the second motion vector, performing motion compensation on the respective position of each target pixel in the second image to obtain the position of each pixel in the fourth image corresponding to each target pixel in the second image; and setting the pixel value of each pixel in the fourth image after motion compensation based on the pixel value of each target pixel in the second image, the relationship between the respective position of each target pixel in the second image and the position of the target object shadow mask mark, and the relationship between the position of each target pixel after motion compensation in the fourth image and the position marked in the target object shadow mask, thereby generating the fourth image.

[0165] It should be understood that the method for obtaining the second motion vector can be similar to the method for obtaining the first motion vector, and will not be elaborated here.

[0166] As an optional embodiment, setting the pixel value of each pixel corresponding to each target pixel in the fourth image after motion compensation includes: if the position of the target pixel in the first image belongs to the position marked in the target object shadow mask, or if the position of the target pixel in the first image does not belong to the position marked in the target object shadow mask, but the position of the target pixel in the fourth image after motion compensation belongs to the position marked in the target object shadow mask, setting the pixel value of the pixel corresponding to the target pixel in the fourth image after motion compensation to a first preset value, the first preset value is used to indicate that the pixel value needs to be modified; if the position of the target pixel in the first image does not belong to the position marked in the target object shadow mask, and the position of the target pixel in the fourth image after motion compensation also does not belong to the position marked in the target object shadow mask, setting the pixel value of the pixel corresponding to the target pixel in the fourth image after motion compensation to the pixel value of the target pixel.

[0167] For example, based on the pixel value of each target pixel in the first image, the relationship between the position of each target pixel in the first image and the position of the target object shadow mask mark, and the relationship between the position of each target pixel in the fourth image after motion compensation and the position of the target object shadow mask mark, the pixel value of each pixel corresponding to the target pixel in the fourth image after motion compensation is set, and the specific process of generating the fourth image can be as described in the above method 700, which will not be repeated here.

[0168] In this embodiment, the electronic device can traverse each target pixel in the first image, perform motion compensation based on the first motion vector to obtain the fourth image, and set the pixel values ​​of each target pixel in the fourth image after motion compensation based on the pixel values ​​of each target pixel in the first image, the relationship between the position of each target pixel in the first image and the position of the target object shadow mask mark, and the relationship between the position of each target pixel in the fourth image after motion compensation and the position of the target object shadow mask mark, so as to obtain the fourth image including pixels with pixel values ​​to be modified, which is beneficial for obtaining the third shadow in the third image based on the first shadow in the first image and the second shadow in the second image.

[0169] As an optional embodiment, the method further includes: acquiring a second motion vector, the second motion vector being used to represent the displacement between the position of the same target in the second image and its position in the third image; modifying the pixel value of a pixel whose pixel value is to be modified in the fourth image based on the pixel values ​​of each pixel in the first image and the pixel values ​​of each pixel in the second image, including: acquiring the pixel whose pixel value is to be modified in the fourth image; determining the position of the pixel corresponding to the pixel whose pixel value is to be modified in the first image based on the first motion vector; determining the position of the pixel corresponding to the pixel whose pixel value is to be modified in the second image based on the second motion vector; if the position of the pixel whose pixel value is to be modified in the fourth image belongs to the position marked in the target object's shadow mask: if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified belongs to the position marked in the target object's shadow mask, then modifying the pixel value of the pixel whose pixel value is to be modified to the pixel value of the pixel corresponding to the pixel in the first image, the pixel whose pixel value is to be modified is included in the first region, and the pixel corresponding to the pixel whose pixel value is to be modified in the first image is included in the third region. If the pixel in the second image corresponding to the pixel whose pixel value is to be modified is located at a position marked in the target object's shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel in the second image. The pixel whose pixel value is to be modified is included in the second region, and the pixel corresponding to the pixel whose pixel value is to be modified is included in the fourth region. If the pixel in the first image corresponding to the pixel whose pixel value is to be modified is not located at a position marked in the target object's shadow mask, and the pixel in the second image corresponding to the pixel whose pixel value is to be modified is also not located at a position marked in the target object's shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to any of the following: the pixel value of the pixel within a preset radius of the location of the pixel whose pixel value is to be modified is located at a position marked in the target object's shadow mask; the pixel value of the pixel within a preset radius of the location of the pixel whose pixel value is to be modified is located at a position marked in the target object's shadow mask; or a second preset value.

[0170] In this embodiment, if the pixel whose pixel value needs to be modified in the fourth image is located at a position marked in the target object's shadow mask, then the pixel should be filled with the shadow color. If a pixel corresponding to the pixel in the first image (or the second image) is found at a position marked in the target object's shadow mask (i.e., the pixel belongs to the first shadow, and its pixel value corresponds to the shadow color), then the pixel value of that pixel is filled into the pixel to be modified. In this way, the pixel values ​​of the pixels whose pixel values ​​need to be modified in the fourth image that are located at positions marked in the target object's shadow mask come from the first shadow of the first image and / or the second shadow of the second image. The pixel values ​​of the pixels whose pixel values ​​need to be modified in the fourth image can be understood as being copied from the corresponding pixels in the first shadow of the first image and / or the second shadow of the second image, thus avoiding texture mismatch and improving the user experience.

[0171] Furthermore, if the position of the pixel whose pixel value needs to be modified in the first image (or the second image) does not belong to the position marked in the target object's shadow mask, it means that the pixel whose pixel value needs to be modified cannot find a shadow color in the first image (or the second image). These pixels could be, for example, […]. Figure 11 Regions ⑤ and ⑥ in the predicted frames excluding character shadows, shown in the diagram, have small differences between each frame. These regions occupy a very small proportion of the overall image and have minimal impact on the global picture. Therefore, the pixel values ​​to be modified in these regions can be the pixel values ​​of the corresponding pixels in the first image that belong to the marked positions in the target object's shadow mask (e.g., the pixel value of the closest pixel in the first image to the corresponding pixel and belonging to the marked positions in the target object's shadow mask), the pixel values ​​of the corresponding pixels in the second image that belong to the marked positions in the target object's shadow mask, or any of the second preset values, to optimize the display effect of the third shadow in the third image. The second preset value can be, for example, any grayscale value. In game applications, the second preset value can be, for example, the average pixel value of shadow areas historically stored on the electronic device, or any pixel value of historically stored shadow areas.

[0172] In addition, the pixel values ​​of the pixels to be modified in regions ⑤ and ⑥ can also be: the pixel values ​​of the corresponding pixels in the first image, or the pixel values ​​of the corresponding pixels in the second image. This application does not make any specific limitations on this.

[0173] Optionally, the second motion vector may be, for example, the aforementioned motion vector 2, but this application does not limit it to this.

[0174] Optionally, the preset radius may be less than or equal to a certain threshold, and this application does not limit this threshold.

[0175] As an optional embodiment, if the position of the pixel whose pixel value is to be modified in the fourth image does not belong to the position marked in the target object's shadow mask: if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the target object's shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the first image; if the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the target object's shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the second image.

[0176] For example, in the fourth image, the pixel whose pixel value needs to be modified is located at a position that does not belong to the position marked in the target object's shadow mask. For example, it could be... Figure 11 The pixels in region ④ of the prediction frame excluding the shadow of the person are not located at the positions marked in the target object shadow mask in the fourth image. That is, these pixels do not belong to the third shadow. In this embodiment, for these pixels, the corresponding pixels that do not belong to the first or second shadow are found in the first or second image. The pixel values ​​of these pixels are modified to match the pixel values ​​of the corresponding pixels that do not belong to the first or second shadow. This is beneficial for modifying the pixel values ​​of pixels in the fourth image whose positions do not belong to the positions marked in the target object shadow mask, thus improving the fourth image and obtaining the third image.

[0177] Optionally, if the position of the pixel whose pixel value is to be modified in the fourth image does not belong to the position marked in the target object's shadow mask: if the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified in the first image belongs to the position marked in the target object's shadow mask, and the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified in the second image belongs to the position marked in the target object's shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to any one of the following: the pixel value of a pixel within a preset radius of the position of the pixel corresponding to the pixel in the first image that does not belong to the position marked in the target object's shadow mask, the pixel value of a pixel within a preset radius of the position of the pixel corresponding to the pixel in the second image that does not belong to the position marked in the target object's shadow mask, or a third preset value.

[0178] The third preset value can be any value that allows a pixel to display a non-shadow color. In game application scenarios, the third preset value can be the average pixel value of the non-shadow area stored in the history of the electronic device, or any pixel value of the non-shadow area stored in the history. This application does not limit this.

[0179] As an optional embodiment, obtaining the pixel point whose pixel value is to be modified in the fourth image includes: traversing each pixel point in the fourth image and obtaining the pixel point whose pixel value is a first preset value as the pixel point whose pixel value is to be modified.

[0180] For example, the method for marking pixels whose pixel values ​​need to be modified in the fourth image can be to set a marker value α in any one of its pixel channels (Red, Green, Blue, and Alpha), and / or to write a motion vector (or pixel coordinates or any value that can be distinguished from non-pixel values ​​to be modified) in any one of the RGBA channels. Therefore, for example, when the electronic device obtains pixels whose pixel values ​​need to be modified in the fourth image, it can traverse each pixel in the fourth image, check whether there is a marker value α in the RGBA channel of each pixel, and obtain the pixels with the marker value α as the pixels whose pixel values ​​need to be modified.

[0181] In this way, the traversal method is beneficial to obtain all the pixels in the fourth image whose pixel values ​​need to be modified, which is beneficial to the fact that the pixel values ​​of these pixels can be modified, which is beneficial to the reduction of the possibility of missing pixel values ​​in the fourth image, and which is beneficial to the improvement of the completeness of the final third image.

[0182] As an optional embodiment, obtaining the target object shadow mask includes: obtaining a fifth image, which is the image at the start of drawing the first shadow during the generation process of the first image; obtaining a sixth image, which is the image at the end of drawing the first shadow during the generation process of the first image; subtracting the pixel values ​​of each pixel in the sixth image and the fifth image respectively, and marking the positions of pixel points whose pixel difference after subtraction is not zero, and / or marking the positions of pixel points whose pixel difference after subtraction is zero, to obtain the target object shadow mask.

[0183] In this embodiment, during the generation of the first image (or second image), a fifth image at the start of drawing the first shadow and a sixth image at the completion of drawing the first shadow can be obtained. The pixel values ​​of each pixel in the sixth image and the fifth image are subtracted accordingly, and the positions of pixels whose pixel difference after subtraction is not zero are marked, and / or the positions of pixels whose pixel difference after subtraction is zero are marked, thus obtaining the target object shadow mask. This method does not affect the generation process of the first image (or second image) and is beneficial for the normal output of the first image (or second image).

[0184] The target object's shadow mask can be, for example, the human shadow mask image described in the above embodiments, and its acquisition process and results can be, for example, as follows: Figure 6 As shown.

[0185] The image processing method of the present application embodiments has been described above. The apparatus for performing the above method provided in the present application embodiments is described below. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced by each other, and the related apparatus provided in the present application embodiments can perform the steps in the above method.

[0186] Figure 13 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown. For example... Figure 13 As shown, the electronic device may include a processor 1310, a memory 1320, a display screen 1330, a sensor module 1340, a universal serial bus (USB) interface 1350, a charging management module 1360, a power management module 1361, a battery 1362, an image processor 1370, etc.

[0187] Processor 1310 may include one or more processing units, such as a central processing unit (CPU), application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0188] The memory 1320 can be used to store computer executable program code, which includes instructions, such as the program code corresponding to the method described above in the embodiments of this application.

[0189] The display screen 1330 is used to display images, videos, etc. The display screen 1330 includes a display panel. In some embodiments, the electronic device may include one or N display screens 1330, where N is a positive integer greater than 1. The electronic device implements display functions through a GPU, the display screen 1330, and an application processor, etc. In this embodiment, the display screen can be used to sequentially display a first image, a third image, and a second image.

[0190] The image processor 1370 can be used to draw a first image and / or a second image based on drawing instructions from an application.

[0191] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0192] The image processing method provided in this application embodiment can be applied to electronic devices with image display functions, and the software structure of the electronic device can be as follows: Figure 4 As shown, the hardware structure can be as follows Figure 13 As shown, the specific form of the electronic device can be referred to the above description, and will not be repeated here.

[0193] Figure 14 An exemplary schematic diagram of a chip structure provided in an embodiment is shown. Chip 1400 includes one or more (including two) processors 1401, communication lines 1402, communication interfaces 1403, and memory 1404.

[0194] In some implementations, memory 1404 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0195] The methods described in the embodiments of this application can be applied to, or implemented by, processor 1401. Processor 1401 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the hardware of processor 1401 or by instructions in software form. Processor 1401 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 1401 can implement or execute the various processing-related methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0196] The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read-only memory (EEPROM). This storage medium is located in memory 1404, and processor 1401 reads information from memory 1404 and, in conjunction with its hardware, completes the steps of the above method.

[0197] The processor 1401, memory 1404 and communication interface 1403 can communicate with each other via communication line 1402.

[0198] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0199] In the embodiments of this application, the chip 1400 may also be a chip system, such as a system on chip (SOC), and this application does not limit it in this way.

[0200] This application provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and stores computer program code, including computer instructions; one or more processors invoke the computer instructions to cause the electronic device to perform the methods described in the above embodiments.

[0201] This application provides a chip or chip system. The chip or chip system is applied to an electronic device and includes one or more processors. These processors invoke computer instructions to cause the electronic device to execute the methods described in the above embodiments. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0202] This application also provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When these computer instructions are executed on an electronic device, they cause the electronic device to perform the methods described in the above embodiments. The methods described in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted on the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0203] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data using laser optics. Combinations of the above should also be included within the scope of computer-readable media.

[0204] This application provides a computer program product, which includes computer program code. When the computer program code is run on an electronic device, the electronic device performs the method described in the above embodiments.

[0205] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0206] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized by, Applied to electronic devices, including: Acquire a first image and a second image, wherein the first image includes a target object and a first shadow of the target object, and the second image includes the target object and a second shadow of the target object; A third image is generated based on the first image and the second image; wherein, the third image includes the target object and a third shadow of the target object, the area occupied by the third shadow includes a first region and a second region, the pixel value of the first region is obtained based on the pixel value of the third region in the first shadow, and the pixel value of the second region is obtained based on the pixel value of the fourth region in the second shadow; the first region and the third region are the same region in the same coordinate system, and the second region and the fourth region are the same region in the same coordinate system; The first image, the third image, and the second image are displayed sequentially.

2. The method of claim 1, wherein, The step of generating a third image based on the first image and the second image includes: A first motion vector is obtained, which represents the displacement between the position of the same target in the first image and its position in the third image; Motion compensation is performed on the first image based on the first motion vector to generate a fourth image. The fourth image includes pixels whose pixel values ​​are to be modified, including pixels in the first region and pixels in the second region. Based on the pixel values ​​of each pixel in the first image and the pixel values ​​of each pixel in the second image, the pixel values ​​of the pixels whose pixel values ​​need to be modified in the fourth image are modified to generate the third image.

3. The method of claim 2, wherein, The method further includes: Obtain the target object shadow mask, which is used to mark the position of the target object's shadow in the image; Motion compensation is performed on the first image based on the first motion vector to generate a fourth image, including: Based on the first motion vector, motion compensation is performed on the position of each target pixel in the first image to obtain the position of each pixel in the fourth image corresponding to each target pixel in the first image; Based on the pixel value of each target pixel in the first image, the relationship between the position of each target pixel in the first image and the position of the target object shadow mask mark, and the relationship between the position of each target pixel in the fourth image after motion compensation and the position of the target object shadow mask mark, the pixel value of each pixel corresponding to each target pixel in the fourth image after motion compensation is set, and the fourth image is generated.

4. The method of claim 3, wherein, Setting the pixel values ​​of each target pixel in the fourth image after motion compensation includes: If the position of the target pixel in the first image belongs to the position marked in the target object shadow mask, or if the position of the target pixel in the first image does not belong to the position marked in the target object shadow mask, but the position of the target pixel in the fourth image after motion compensation belongs to the position marked in the target object shadow mask, the pixel value of the corresponding pixel in the fourth image after motion compensation is set to a first preset value. The first preset value is used to indicate that the pixel value needs to be modified. If the position of the target pixel in the first image does not belong to the position marked in the target object shadow mask, and the position of the target pixel in the fourth image after motion compensation also does not belong to the position marked in the target object shadow mask, then the pixel value of the pixel corresponding to the target pixel in the fourth image after motion compensation is set to the pixel value of the target pixel.

5. The method according to any one of claims 2 to 4, characterized in that, The method further includes: A second motion vector is obtained, which represents the displacement between the position of the same target in the second image and its position in the third image; The step of modifying the pixel values ​​of the pixels to be modified in the fourth image based on the pixel values ​​of each pixel in the first image and the pixel values ​​of each pixel in the second image includes: Obtain the pixel whose pixel value needs to be modified in the fourth image, and determine the position of the corresponding pixel in the first image based on the first motion vector; Based on the second motion vector, determine the position of the pixel in the second image corresponding to the pixel whose pixel value needs to be modified. If the position of the pixel whose pixel value needs to be modified in the fourth image belongs to the position marked in the target object's shadow mask: If the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified is the position marked in the shadow mask of the target object, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the first image. The pixel whose pixel value is to be modified is included in the first region, and the pixel corresponding to the pixel whose pixel value is to be modified in the first image is included in the third region. If the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified is the position marked in the shadow mask of the target object, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the second image. The pixel whose pixel value is to be modified is included in the second region, and the pixel corresponding to the pixel whose pixel value is to be modified in the second image is included in the fourth region. If the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the target object shadow mask, and the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the target object shadow mask, then the pixel value of the pixel whose pixel value is to be modified is modified to any one of the following: the pixel value of the pixel within a preset radius of the position of the pixel corresponding to the pixel in the first image that belongs to the position marked in the target object shadow mask; the pixel value of the pixel within a preset radius of the position of the pixel corresponding to the pixel in the second image that belongs to the position marked in the target object shadow mask; or a second preset value.

6. The method according to claim 5, characterized in that, If the position of the pixel whose pixel value needs to be modified in the fourth image does not belong to the position marked in the target object's shadow mask: If the position of the pixel in the first image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the shadow mask of the target object, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel in the first image. If the position of the pixel in the second image corresponding to the pixel whose pixel value is to be modified does not belong to the position marked in the shadow mask of the target object, then the pixel value of the pixel whose pixel value is to be modified is modified to the pixel value of the pixel corresponding to the pixel whose pixel value is to be modified in the second image.

7. The method according to claim 5 or 6, characterized in that, The step of obtaining the pixels whose pixel values ​​in the fourth image need to be modified includes: Traverse each pixel in the fourth image and select the pixel whose pixel value is a first preset value as the pixel whose pixel value is to be modified.

8. The method according to claim 3, characterized in that, The step of obtaining the target object shadow mask includes: Obtain a fifth image, which is the image generated during the process of creating the first image, at the moment when the first shadow begins to be drawn; Obtain the sixth image, wherein the fifth image is the image generated during the first image generation process, at the point when the first shadow is drawn; Subtract the pixel values ​​of each pixel in the sixth image and the fifth image respectively, and mark the positions of pixels whose pixel difference after subtraction is not zero, and / or mark the positions of pixels whose pixel difference after subtraction is zero, to obtain the target object shadow mask.

9. The method according to any one of claims 1 to 8, characterized in that, The electronic device includes a game application with a shadow function enabled, and the first image and the second image are images drawn based on the drawing instructions of the game application.

10. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 9.

11. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 9.

Images