Image processing method and electronic equipment

Abstract

The embodiment of the invention provides an image processing method and electronic equipment, and the method comprises the steps: obtaining a first flow diagram and a first texture map in response to a first operation of a user, the first flow diagram being a texture map containing preset vector field information, and the first texture map being a seamless texture map generated according to noise; respectively executing a first processing process at a plurality of first sampling moments; the first processing process comprises the steps that at a first target moment, N1 sets of texture offset sampling along with time are carried out on a first texture map, sampling disturbance is carried out through a first flow diagram in the sampling process, N1 first simulation maps corresponding to the first target moment are obtained, N1 is an integer larger than 1, and the first target moment is any one of multiple first sampling moments; fusing the N1 first simulation maps to obtain a first frame image corresponding to the first target moment; and displaying the first frame image. The method can reduce the performance overhead of the electronic equipment.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to an image processing method and electronic equipment. Background Art

[0002] In the application of electronic devices, dynamic effects such as caustics and bubbles are used in many scenarios. These dynamic effects can increase the vividness of the interface and enhance the user's visual experience.

[0003] Taking caustics as an example, in traditional technology, the dynamic effects of caustics are generally displayed by using complex grid models, which results in a relatively high performance overhead for electronic devices. Summary of the Invention

[0004] The present application provides an image processing method and electronic device, which can reduce performance overhead.

[0005] In a first aspect, the present application provides an image processing method, which is performed by an electronic device. The method includes: in response to a first user operation, obtaining a first flow map and a first texture map, wherein the first flow map is a texture map containing preset vector field information, and the first texture map is a seamless texture map generated based on noise; and performing a first processing process at multiple first sampling moments respectively;

[0006] The first processing process includes: at a first target moment, performing N1 groups of texture offset sampling over time on the first texture map, and performing sampling perturbation through the first flow direction map during the sampling process to obtain N1 first simulation maps corresponding to the first target moment, where N1 is an integer greater than 1, and the first target moment is any one of multiple first sampling moments; fusing the N1 first simulation maps to obtain a first frame image corresponding to the first target moment; and displaying the first frame image.

[0007] It can be understood that the first processing process is executed separately at multiple first sampling moments, that is, each first sampling process repeats the first processing process. Each time the first processing process is executed once, a first frame image is generated and displayed. The first processing process is a process of performing texture offset sampling over time based on the first texture map (that is, the process of rendering the first frame image). As time goes by, each first sampling moment is different, and thus the texture details of the first frame image obtained by executing the first processing process are different. In this way, by displaying each first frame image frame by frame, simulating the change of texture in the first texture map over time, the dynamic effect corresponding to the first texture map can be achieved.

[0008] Optionally, the first texture map can be an image with a preset display effect, such as a caustic effect, a foam effect, or a wave effect. For example, if the first texture map has a foam effect, the first frame image also has the foam effect, and the texture details of the first frame images obtained at different sampling times are different. In this way, by sequentially displaying multiple first frame images, the evolution of foam over time can be simulated, thereby achieving a dynamic foam effect.

[0009] Optionally, the texture map generated based on noise refers to a texture map generated by a noise algorithm and / or noise data.

[0010] The image processing method provided by the present application can present a dynamic image corresponding to the display effect of the first texture map based on the first texture map. The entire process does not require animation or complex mesh models, thus reducing the performance overhead of the electronic device and improving the operating efficiency of the electronic device. In addition, the first texture map is a map generated based on noise, which is random and relatively realistic and natural. Moreover, the first texture map is a seamless texture map, without obvious seams or discontinuities between textures. Therefore, in the method provided by the embodiment of the present application, the first simulated image obtained by texture offset sampling based on the first texture map is relatively realistic and natural, without obvious seams or discontinuities, and has a good image quality. This in turn makes the subsequently generated first frame image realistic and natural, and the resulting dynamic image quality is good, improving the user experience. In addition, the method provided by the embodiment of the present application perturbs the texture sampling of the first texture map through a first flow map. The first flow map includes preset vector field information, which can make the first simulated map exhibit quantitative flow characteristics, further increase the randomness of the texture changes over time in the first texture map, make the simulated texture changes more realistic and natural, and thus make the subsequent dynamic image quality better, improving the user experience.

[0011] In one possible implementation, at a first target moment, N1 groups of texture offset sampling are performed on the first texture map over time, and sampling perturbations are performed through the first flow map during the sampling process to obtain N1 first simulation maps corresponding to the first target moment, including: at the first target moment, texture offset sampling is performed on each texture coordinate in the first flow map over time to obtain a first offset corresponding to each texture coordinate; each texture coordinate is offset according to the first offset corresponding to each texture coordinate to obtain a first target coordinate corresponding to each texture coordinate; according to the first target moment, N1 groups of texture offset sampling are performed on each first target coordinate in the first texture map over time to obtain N1 first simulation maps.

[0012] In this implementation, the first flow map is sampled over time using a texture offset to obtain different first offsets, and thus different vector field information, at different times. This sampling perturbation, performed using different vector field information at different times, further increases the randomness of the texture changes over time in the first texture map, making the simulated texture changes more realistic and natural. This, in turn, enhances the subsequent dynamic image quality and improves the user experience.

[0013] In a possible implementation, N1 first simulation maps are fused to obtain a first frame image corresponding to a first target moment, including: performing weighted summation on the texture values ​​of each target coordinate in the N1 first simulation maps to obtain the first frame image.

[0014] In one possible implementation, the first processing process stops after executing for a preset time. In this way, the dynamic effect of the texture in the picture (such as the caustic dynamic effect) stops after a period of time, saving power consumption of the electronic device.

[0015] In a possible implementation, the time interval between two adjacent first sampling moments is equal to a refresh period corresponding to a refresh rate of the electronic device.

[0016] That is, the first processing is performed once every refresh cycle to obtain a first frame image, which is consistent with the refresh rate of the electronic device, can improve the image display effect and save power consumption.

[0017] In one possible implementation, the method further includes: receiving a touch operation performed by a user on a first interface, the first interface including a first frame image; in response to the touch operation, obtaining a touch trajectory corresponding to the touch operation; generating a second flow map based on the touch trajectory, the second flow map being a texture map including vector field information corresponding to the touch trajectory; and performing a second processing process at a plurality of second sampling moments respectively;

[0018] The second processing process includes: at the second target moment, performing N2 groups of texture offset sampling over time on the first texture map, and performing sampling perturbation through the second flow map during the sampling process to obtain N2 second simulation maps corresponding to the second target moment, where N2 is an integer greater than 1, and the second target moment is any one of the multiple second sampling moments; fusing the N2 second simulation maps to obtain a second frame image corresponding to the second target moment; and displaying the second frame image.

[0019] The second process is similar to the first, differing in that it uses texture offset sampling based on the second flow map. The second flow map is a texture map generated based on the touch trajectory, containing the corresponding vector field information. Therefore, the texture flow in the generated second frame is correlated with the user's touch trajectory, enhancing the realism of the dynamic effect and making the interaction more engaging.

[0020] In one possible implementation, at the second target moment, N2 groups of texture offset sampling over time are performed on the first texture map, and sampling perturbations are performed through the second flow map during the sampling process to obtain N2 second simulation maps corresponding to the second target moment, including: at the second target moment, texture offset sampling is performed on each texture coordinate in the second flow map to obtain a second offset corresponding to each texture coordinate; each texture coordinate is offset according to the second offset corresponding to each texture coordinate to obtain a second target coordinate corresponding to each texture coordinate; according to the second target moment, N2 groups of texture offset sampling are performed on each second target coordinate in the first texture map to obtain N2 second simulation maps.

[0021] The beneficial effects of this implementation are similar to those corresponding to generating the first simulation map, and will not be described in detail.

[0022] In one possible implementation, at the second target moment, after performing texture offset sampling on each texture coordinate in the second flow map over time and obtaining the second offset corresponding to each texture coordinate, the method further includes: determining the change gradient of the second offset corresponding to the first texture coordinate within a preset time period before the first sampling moment to obtain the first change gradient, where the first texture coordinate is any texture coordinate in the second flow map.

[0023] In a possible implementation, the preset time period is a time period between a sampling moment before the first sampling moment and the first sampling moment.

[0024] In one possible implementation, N2 is an integer greater than 2, and N2 second simulated maps are fused to obtain a second frame image corresponding to the second target moment, including: if the first change gradient is greater than a preset gradient threshold, weighted summing the texture values ​​corresponding to the first texture coordinate in the N2 second simulated maps to obtain the texture value corresponding to the first texture coordinate in the second frame image; if the first change gradient is less than or equal to the preset gradient threshold, weighted summing the texture values ​​corresponding to the first texture coordinate in the N3 second simulated maps to obtain the texture value corresponding to the first texture coordinate in the second frame image, where N3 is an integer greater than 1 and less than N2.

[0025] In the above-mentioned implementation methods, if the first change gradient corresponding to the first texture coordinate is large (greater than the preset gradient threshold), it means that the first texture coordinate is likely to contain the user's touch track, so the texture values ​​corresponding to the first texture coordinate in a larger number (N2 sheets) of the second simulated maps are superimposed to obtain the texture value corresponding to the first texture coordinate. This can make the texture at the first texture coordinate denser. If the first change gradient corresponding to the first texture coordinate is small (less than or equal to the preset gradient threshold), it means that the first texture coordinate is likely not to contain the user's touch track, so the texture values ​​corresponding to the first texture coordinate in a smaller number (N3 sheets) of the second simulated maps are superimposed to obtain the texture value corresponding to the first texture coordinate. This can make the texture at the first texture coordinate sparser. Through the above process, texture effects of different densities in different areas can be achieved according to the user's touch track, thereby improving the fun and realism of the dynamic effect and thus improving the user experience.

[0026] In one possible implementation, N1 is smaller than N2.

[0027] In other words, during the first processing step, a smaller number of first simulated maps are selected for overlay. Since the first process corresponds to a scenario where the user does not perform a touch operation on the first interface, there is no need to perform image fusion based on the user's touch trajectory. Directly selecting a smaller number of first simulated maps for overlay simplifies algorithm operation efficiency and saves power consumption in the electronic device.

[0028] In a possible implementation, the first interface is a lock screen interface or a desktop.

[0029] In a possible implementation, obtaining the first flow map and the first texture map includes: obtaining the first flow map and the first texture map from a server or a memory of an electronic device.

[0030] That is, the first flow map and the first texture map can be generated offline without real-time generation, which further saves the performance overhead of the electronic device and further improves the operating efficiency of the electronic device.

[0031] In a possible implementation, the first operation is an operation of setting the first wallpaper as a lock screen wallpaper and / or desktop wallpaper, and the first wallpaper is a dynamic wallpaper corresponding to the first texture map.

[0032] In a possible implementation, the first texture map has a caustic effect.

[0033] In a second aspect, the present application provides a device, which is included in an electronic device and has the function of implementing the electronic device behavior described in the first aspect and possible implementations of the first aspect. The function can be implemented through hardware or through hardware executing corresponding software implementations. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a receiving module or unit, a processing module or unit, etc.

[0034] In a third aspect, the present application provides an electronic device, which includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute any one of the methods in the technical solution of the first aspect.

[0035] In a fourth aspect, the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the method in the first aspect and any possible implementation thereof.

[0036] Optionally, the chip system also includes a memory, and the memory is connected to the processor via circuits or wires.

[0037] Further optionally, the chip system also includes a communication interface.

[0038] In a fifth aspect, the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on an electronic device, the electronic device executes any one of the methods in the technical solution of the first aspect.

[0039] In a sixth aspect, the present application provides a computer program product, which includes: a computer program code, which, when the computer program code runs on an electronic device, enables the electronic device to execute any one of the methods in the technical solution of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of a caustic effect provided in an embodiment of the present application;

[0041] Figure 2 This is a schematic diagram of a simulation of the water surface caustic effect provided by an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0043] Figure 4 This is a software structure diagram of an electronic device provided in an embodiment of the present application;

[0044] Figure 5 This is a schematic diagram of an interface change provided in an embodiment of the present application;

[0045] Figure 6 This is a schematic diagram of a lock screen interface provided in an embodiment of the present application;

[0046] Figure 7 This is another example of interface change diagram provided in an embodiment of the present application;

[0047] Figure 8 This is a flowchart of an image processing method provided in an embodiment of the present application;

[0048] Figure 9 is a schematic diagram of an example of a noise caustic seamless map provided in an embodiment of the present application;

[0049] Figure 10 This is a schematic diagram of an example of repeatedly splicing noise caustics seamless maps provided in an embodiment of the present application;

[0050] Figure 11 This is a schematic diagram of an example of flowmap and vector field information provided in an embodiment of the present application;

[0051] Figure 12 is a schematic diagram of an example of a caustic simulation map provided in an embodiment of the present application;

[0052] Figure 13 is a schematic diagram of another example of a caustic simulation map provided in an embodiment of the present application;

[0053] Figure 14 1 is a schematic diagram of different caustic frame images obtained by using different superposition methods according to an embodiment of the present application;

[0054] Figure 15 is a flowchart of another image processing method provided in an embodiment of the present application;

[0055] Figure 16 This is another example of an interface change diagram provided in an embodiment of the present application;

[0056] Figure 17 This is a flowchart of another image processing method provided in an embodiment of the present application;

[0057] Figure 18 This is a flowchart of another image processing method provided in an embodiment of the present application;

[0058] Figure 19 This is a schematic diagram of an example of foam dynamic effect provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0060] In the following, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include one or more of the features.

[0061] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of the present application include a particular feature, structure, or characteristic described in conjunction with that embodiment. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in different places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0062] To better understand the embodiments of the present application, the terms or concepts that may be involved in the embodiments are explained below.

[0063] 1. Caustics

[0064] Caustics refer to the bright spots or light and shadow effects formed on other surfaces after light is refracted or reflected through certain media (such as water surface, glass, etc.). When the medium is water surface, the caustics formed can be called water surface caustics; when the medium is glass, the caustics formed can be called glass caustics. For example, Figure 1 A schematic diagram of a caustic effect provided in an embodiment of the present application.

[0065] Caustics can oscillate as the medium or light changes. For example, caustics on water will oscillate as the water ripples. In other words, caustics can be dynamic.

[0066] 2. Seamless Texture

[0067] Seamless mapping means that when the map is tiled (repeatedly spliced) in various directions (such as horizontally and vertically), there will be no obvious seams or discontinuities between the textures.

[0068] 3. Flowmap

[0069] A flowmap is a texture data structure that records 2D vector field information. Imagine a 2D plane, where each point corresponds to a vector that points in the direction of the point's next movement. By recording these vectors using texture values, a flowmap is created. Texture values ​​can include color values, which typically consist of two channels: red (R) and green (G).

[0070] 4. Textures, Texture Data, and Texture Values

[0071] Texture refers to the details and features of an object's surface. It is composed of texture primitives and their arrangement patterns. In computer graphics, texture data is a set of data that represents the details and features of an object's surface. Texture data can be an image, a pattern, or other forms of data. When texture data is an image, it is called a texture map. Texture data includes texture values ​​corresponding to each texture coordinate. Texture values ​​can represent information such as the texture's color, brightness, and transparency. By applying texture values ​​to an object's surface, you can enhance the object's realism and detail.

[0072] 5. Texture Coordinates

[0073] In computer graphics, in order to map a texture (which can be a 2D texture, 1D texture, or 3D texture, etc.) onto a graphic, it is necessary to associate a texture coordinate (also called texture mapping coordinate) with each vertex of the graphic. By assigning texture coordinates to each vertex, it is possible to indicate which part of the texture map to sample the texture value (for example, the color), and then perform fragment interpolation on other fragments of the graphic, thereby achieving the mapping of the texture onto the graphic.

[0074] Texture coordinates can be represented by x- and y-axis values. In a 2D texture image, the x- and y-axis range from 0 to 1. In other words, in texture coordinates, the x- and y-coordinates both range from [0, 1].

[0075] The following describes the application scenarios of the image processing method provided in the embodiments of the present application and the technical problems faced by the present application.

[0076] The image processing method provided in the embodiments of the present application can be used to display dynamic effects based on light or water. The dynamic effects based on light or water include, but are not limited to, caustic dynamic effects, wave dynamic effects, waterfall flow effects, foam dynamic effects, etc. The caustic dynamic effects can be water surface caustic dynamic effects or glass caustic dynamic effects.

[0077] Taking the caustic dynamic effect on the water surface as an example, in some scenarios, the application can present realistic pictures to users through the caustic dynamic effect, thereby simulating real scenes and improving the user experience. For example, in a certain game application (application, APP), it is necessary to display an interface diagram that simulates a seaside scene. In this case, the caustic dynamic effect can be used to make the picture more beautiful and the simulated seaside scene more realistic. For another example, in the wallpaper APP, wallpapers with caustic dynamic effects (hereinafter referred to as caustic dynamic wallpapers) are provided to users. Users can choose caustic dynamic wallpapers as lock screen wallpapers or desktop wallpapers, which can not only improve the user's visual experience, but also increase the fun. For example, Figure 2 This is a schematic diagram of a simulation of a water surface caustic effect provided in an embodiment of the present application. Specifically, this diagram is a frame image of a water surface caustic dynamic effect generated by an electronic device. Figure 2 It can be seen that the dynamic effect of water surface caustics can improve the realism of the picture and enhance the user's visual experience.

[0078] It's understandable that these light- or water-based dynamic effects can be implemented in a variety of ways, including through vertex animation, mesh models, and real-time noise map calculations. However, because these dynamic effects are complex, often involving detailed, irregular variations and potentially involving simulations of light propagation, refraction, and reflection, they often incur significant performance overhead for electronic devices.

[0079] The image processing method provided in the embodiments of the present application is intended to solve this problem.

[0080] The image processing method provided in the embodiments of the present application can be applied to electronic devices that can install application programs, such as mobile phones, tablet computers, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0081] For example, Figure 3 1 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0082] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0083] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0084] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0085] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0086] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information. In embodiments of the present application, the GPU and application processor can be used together to implement image processing methods.

[0087] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0088] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0089] Figure 4This is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0090] like Figure 4 As shown, the application package can include applications such as camera, gallery, calendar, call, map, navigation, music, video, etc.

[0091] In the embodiment of the present application, the application package may also include lock screen, game and other applications.

[0092] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0093] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0094] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0095] Content providers are used to store and retrieve data and make it accessible to applications. Data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0096] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0097] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0098] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0099] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0100] In an embodiment of the present application, the application framework layer also includes a graphics rendering module and a layer synthesizer. The graphics rendering module user obtains rendering input data and inputs the rendering input data to the GPU. Optionally, the rendering input data may include texture data, vertex data, index data, material data, lighting data, and the like. Vertex data is used to describe the vertex features of an object. Vertex data may include the position coordinates of the vertex, the color value of the vertex, the normal vector of the vertex, the texture coordinates of the vertex, and the like. Index data is used to describe how to form triangles or other primitives to construct the surface of an object. Material data is used to describe the surface material of an object. Lighting data is used to describe the light source conditions of the environment in which the object is located, including the type of light source, the position, direction, color, and intensity of the light source, and the like.

[0101] The layer compositor may be, for example, SurfaceFlinger, which is used for layer synthesis to obtain a frame image.

[0102] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0103] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0104] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0105] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0106] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0107] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0108] The 3D Graphics Processing Library is a programming interface for graphics rendering. It provides developers with a set of instructions and functions for describing the graphics content and operations to be rendered, thereby implementing GPU functions such as 3D graphics drawing, image rendering, compositing, and layer processing.

[0109] A 2D graphics engine is a drawing engine for 2D drawings.

[0110] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0111] in addition, Figure 4 The module of the hardware layer related to the above software architecture is also shown. Figure 4 As shown, the hardware layer may include a GPU, a display, etc. The graphics rendering module of the application architecture layer may input data, such as rendering input data, to the GPU of the hardware layer. The display may display based on the frame image provided by the layer compositor.

[0112] For ease of understanding, the following examples of this application will be described with Figure 3 and Figure 4 Taking the electronic device with the structure shown as an example, and taking the application scenario of the lock screen wallpaper with dynamic water surface caustic effect as an example, the image processing method provided in the embodiment of the present application is specifically explained in combination with the two usage processes in this application scenario.

[0113] Step 1: Set up the caustics dynamic wallpaper.

[0114] In one embodiment, the user can set the caustics dynamic wallpaper through the wallpaper setting portal. Figure 5 This is a schematic diagram of an interface change example provided in an embodiment of the present application, taking a mobile phone as an example. Figure 5 As shown in FIG. 5( a ), the desktop 501 of the mobile phone includes an icon 5011 for setting the APP. In response to a user's selection operation (eg, a click operation) on the icon 5011 for setting the APP, the mobile phone displays the following: Figure 5 The setting interface 502 shown in FIG. (b) in FIG. The setting interface 502 includes multiple setting items of the mobile phone, such as Bluetooth, WLAN, personal hotspot, etc. The multiple setting items also include a "Desktop and Personalization" setting item 5021. In response to the user's selection operation (such as a click operation) on the "Desktop and Personalization" setting item 5021, the mobile phone can display an interface 503, such as Figure 5 As shown in Figure (c) in the figure. Interface 503 includes multiple personalized setting items such as "Wallpaper" setting item 5031, "Theme" setting item 5032, "Font" setting item, "Icon" setting item, etc. Among them, the "Wallpaper" setting item 5031 is the setting entrance for wallpaper. Optionally, the "Wallpaper" setting item 5031 can be presented in the form of a card, and the card can rotate the preview image of the lock screen interface of the mobile phone and the preview image of the desktop. For example, the "Wallpaper" setting item 5031 in interface 503 currently displays the preview image 5032 of the lock screen interface, and the preview image 5032 can present the effect of the current lock screen wallpaper.

[0115] In the case where the preview image 5032 is displayed in the “Wallpaper” setting item 5031, in response to the user's selection operation (eg, click operation) on the “Wallpaper” setting item 5031, the mobile phone displays the following Figure 5 The editing interface 504 of the lock screen wallpaper shown in FIG (d) of FIG. The editing interface 504 of the lock screen wallpaper includes a "lock screen style" control 5041. In response to the user's selection operation on the "lock screen style" control 5041, the mobile phone displays the following Figure 5 FIG. 5( e ) shows a lock screen display interface 505 . The lock screen display interface 505 includes a variety of lock screen style options, including a caustic dynamic wallpaper option 5051 .

[0116] In response to the user's selection operation (eg, a click operation) on the caustics dynamic wallpaper option 5051, the electronic device renders the caustics dynamic wallpaper to display Figure 5 The lock screen wallpaper preview interface 506 is shown in FIG (f). It can be seen that in the preview image 5061 in the lock screen wallpaper preview interface 506, the lock screen wallpaper is replaced with a caustic dynamic wallpaper. The lock screen wallpaper preview interface 506 includes an "apply" control 5062. In response to the user's selection operation on the "apply" control 5062, the mobile phone sets the lock screen wallpaper to the caustic dynamic wallpaper. When the user locks the screen, the mobile phone displays the following Figure 6 The lock screen interface 601 shown. It should be noted that, Figure 6 The lock screen wallpaper in lock screen interface 601 in the example above displays a static caustic image. However, the lock screen wallpaper can actually display a dynamic caustic effect. That is, the position and density of the caustics in the lock screen wallpaper will change at different times, and the entire caustic image can appear to be shaking. Optionally, the amplitude of the caustic shaking can be reduced over time, and after a period of time, the caustics can stop shaking.

[0117] In another embodiment, the user can also set the caustics dynamic wallpaper through the theme setting portal. The theme setting portal is used to set the lock screen wallpaper, desktop wallpaper and off-screen display content of the electronic device to the same theme.

[0118] For example, Figure 7 This is another example of interface change diagram provided in the embodiment of the present application. Figure 7 As shown in FIG. (a), the mobile phone displays interface 503. The display process and content of interface 503 can be found in the above Figure 5 The embodiments shown will not be described in detail here.

[0119] The "Theme" setting item 5032 in the interface 503 is the theme setting entry. In response to the user's selection operation on the "Theme" setting item 5032, the mobile phone displays the theme setting interface 701, such as Figure 7 As shown in FIG. (b) of FIG. 701, the theme setting interface 701 includes a caustic theme option 7011. In response to the user's selection operation on the caustic theme option 7011, the caustic theme setting interface 702 is entered, as shown in FIG. Figure 7 The caustics theme setting interface 702 displays a preview image 7021 of a lock screen wallpaper (ie, a caustics dynamic lock screen wallpaper) under the caustics theme, a preview image 7022 of a desktop wallpaper under the caustics theme, and the like.

[0120] The setting interface 702 of the caustic theme also includes an application control 7023. In response to the user's selection operation of the application control 7023, the electronic device sets the lock screen wallpaper, desktop wallpaper and off-screen display to a caustic dynamic effect. After setting the lock screen wallpaper, when the phone is in the lock screen state, the phone can display the following Figure 6 The lock screen interface 601 is shown.

[0121] The image processing method provided in the embodiment of the present application is described below in conjunction with the above-mentioned interface change process.

[0122] Figure 8 : is a flowchart of an image processing method provided in an embodiment of the present application, the method comprising:

[0123] S101. In response to a user setting a caustics dynamic wallpaper, the lock screen application sends a rendering instruction to a graphics rendering module. The rendering instruction carries information about a caustics effect image in a to-be-displayed interface and a caustics identifier.

[0124] The operation of setting the caustics dynamic wallpaper (also called the first operation) can be as described above. Figure 5 The selection operation of the application control 5062 in the interface 506 shown in FIG (f) can also be the above Figure 7 The selection operation of the application control 7023 in the caustic setting interface 702 shown in Figure (c) is not specifically limited in this application.

[0125] The rendering instruction is used to instruct the rendering of the interface to be displayed, and the interface to be displayed is, for example, the lock screen interface 601.

[0126] Optionally, the rendering instruction may carry layout information and resource information of the interface to be displayed. The layout information is used to indicate the position, size and other information of the view controls included in the interface to be displayed. The view controls may be images or other controls. Resource information refers to information on view controls (including images) included in the interface to be displayed. In this embodiment, the resource information includes information on the caustic effect map. It can be understood that the electronic device can support users to select a variety of caustic dynamic effects, and the information on the caustic effect map is used to indicate which caustic dynamic effect is in the interface to be displayed. Optionally, the caustic effect map can be any caustic static map in the process of forming the caustic dynamic effect. The information on the caustic effect map can be the name, number or ID of the caustic effect map, etc., which is not limited.

[0127] In addition, in this embodiment, the rendering instruction also carries a caustic flag, which is used to indicate that the interface to be displayed includes a caustic dynamic effect.

[0128] S102. The graphics rendering module responds to the rendering instruction and, when determining that there is a caustic flag in the rendering instruction, obtains a noise caustic seamless map (also called a first texture map) and a preset flowmap (also called a first flow map) corresponding to the information of the caustic effect map.

[0129] Specifically, after receiving the rendering instruction, the graphics rendering module determines that the interface to be displayed includes a caustic dynamic effect based on the caustic identifier, and then further obtains the corresponding noise caustic seamless map based on the caustic effect map. The noise caustic seamless map is used to render the caustic frame image.

[0130] The following is an explanation of the noise caustic seamless map:

[0131] a. Noise Caustics Seamless Map is a texture map with caustics effect. Noise Caustics Seamless Map can be a 2D map.

[0132] b. Noise caustic seamless maps are images generated using noise algorithms and / or noise data. It should be understood that in the field of graphics, noise is not interference or useless signals in the usual sense, but rather random data with certain characteristics. This noise data can be processed and utilized through specific algorithms and rules to generate natural, realistic, and random images. For example, noise algorithms and / or noise data can be used to generate maps of landform features such as mountains, hills, and valleys; for another example, noise algorithms and / or noise data can be used to generate ripple maps; and for another example, noise algorithms and / or noise data can be used to generate caustic maps.

[0133] c. The noise caustic seamless map is a seamless map, that is, when it is tiled (repeatedly spliced) in all directions, there will be no obvious seams or discontinuities between the textures.

[0134] For example, Figure 9 This is a schematic diagram of a noise caustic seamless map provided by an embodiment of the present application. It can be seen that the map has a caustic effect and the texture in the map is random. In addition, the map is repeatedly spliced ​​in all directions, and there will be no obvious seams between the textures. For example, see Figure 10 ,Will Figure 9 The noise caustics seamless map shown is repeated horizontally and vertically with no noticeable seams or discontinuities between the textures.

[0135] Optionally, the electronic device can pre-generate a noise caustic seamless map corresponding to each caustic dynamic effect, establish a correspondence between the information of each noise caustic seamless map and the caustic effect map, and store it in a preset storage path or server in the electronic device. When it is determined that the rendering instruction carries the information of the caustic identifier and the caustic effect map, the correspondence can be obtained from the preset storage path, and the noise caustic seamless map corresponding to the information of the caustic effect map can be searched. In other words, the noise caustic seamless map can be generated and saved offline. In this way, there is no need to generate the noise caustic seamless map in real time, which reduces the performance overhead of the electronic device and improves the operating efficiency of the electronic device.

[0136] The preset flowmap is pre-generated and contains texture data of preset motion direction and size. In other words, the vector field information contained in the preset flowmap is pre-set. Optionally, the preset flowmap can be stored in a preset storage path of the electronic device or in a server. For example, Figure 11 This is a schematic diagram of an example of flowmap and vector field information provided in an embodiment of the present application. Figure 11 Figure (a) is a schematic diagram of flowmap. Figure 11 Figure (b) is Figure 11 Schematic diagram of the vector field information contained in Figure (a). Figure 11 In Figure (b), each arrow represents the movement direction and size corresponding to the point in the flowmap.

[0137] Of course, the graphics rendering module obtains other rendering input data. Other rendering input data refers to other texture data besides noise caustic seamless maps and preset flowmaps, as well as vertex data, material data, etc.

[0138] S103: The graphics rendering module inputs the noise caustic seamless map and the preset flowmap into the GPU.

[0139] Of course, the graphics rendering module also inputs other rendering input data into the GPU.

[0140] Optionally, the graphics rendering module can generate shaders and input various rendering input data to the GPU through the shaders. Shaders are programs used in computer graphics to control the appearance and performance of objects during the rendering process. Shaders can include vertex shaders and fragment shaders. Vertex shaders process vertex data. Fragment shaders process color data, texture data, lighting data, material data, etc.

[0141] S104, GPU performs N1 sets of texture offset sampling on the noise caustic seamless map at the current sampling moment, and in the sampling process, performs sampling perturbation through the preset flowmap to obtain N1 caustic simulation maps corresponding to the current sampling moment. Figure 1 (Also referred to as the first simulation map.) N1 is an integer greater than or equal to 2.

[0142] Optionally, the GPU can pre-set the sampling time for texture offset sampling, and at each sampling time, the noise caustic seamless map is offset over time. Optionally, the sampling time can be set periodically, that is, each sampling time can be determined according to a preset sampling period. The sampling period can, for example, be equal to the refresh period corresponding to the screen refresh rate (i.e., 1 / refresh rate). In this embodiment, the current sampling time is used as an example to illustrate that the current sampling time refers to the time when texture offset sampling is currently performed on the noise caustic seamless map.

[0143] Optionally, in the process of temporal texture offset sampling, different offsets can be selected to obtain different caustic simulation maps to simulate the caustic effects at different moments, that is, to simulate the effect of caustic changes over time. In this embodiment, N1 sets of texture offset sampling with different offsets are performed on the noise caustic seamless map over time to obtain N1 caustic simulation maps. Figure 1 . This N1 caustic simulation sticker Figure 1 It can simulate the effect of caustics changing over time. For example, Figure 12 This is a schematic diagram of a caustic simulation map provided in an embodiment of the present application. Figure 12 Figures (a), (b), (c) and (d) show the caustic simulation images obtained by using different offset values ​​when sampling with time at a certain moment. Figure 1 Schematic diagram (without flowmap sampling disturbance). It can be seen that Figure 12 The caustic effects in (a), (b), (c) and (d) are not the same. For example, the density of the caustic texture is different, and the position of the caustic texture is also different. In this embodiment, N1 sets of caustic simulation maps with different offsets are used. Figure 1 Can simulate similar Figure 12 The time evolution of caustics shown in .

[0144] Sampling perturbation is performed through a preset flowmap, that is, the texture coordinates used for sampling are offset through the vector field information in the flowmap. The preset flowmap can be understood as a texture map that records 2D vector field information. The texture values ​​of each point in the preset flowmap, such as the color values ​​of the R and G channels, can represent the direction and size of the vector field of the point. By perturbing the texture sampling of the noise caustic seamless map through the preset flowmap, the subsequent image can show the characteristics of quantitative flow, further increasing the randomness of the caustic change over time, making the simulated caustic change process more realistic and natural, thereby making the subsequent caustic dynamic effect better and improving the user experience.

[0145] For example, Figure 13 This is a schematic diagram of another example of a caustic simulation map provided in an embodiment of the present application. Figure 12 The difference is, Figure 13 The caustic simulation maps in the figure are the effects after flowmap sampling disturbance. Figure 12 It can be seen that Figure 12 The caustic texture changes of each image in are relatively regular. Figure 13 In , after sampling perturbation, the change of caustic texture is random. For example, Figure 13 Compared with Figure (b) Figure 13 In (a), the caustic texture in region 1301 becomes sparse, while the caustic texture in region 1302 becomes dense. Figure 13 Compared with Figure (c) in Figure 13 In the (b) figure, the caustic texture in the area 1303 becomes sparse. Figure 13 Compared with Figure (d) Figure 13 In Figure (c), the caustic texture in region 1304 becomes dense. This shows that after the flowmap sampling perturbation, the change of caustics is random.

[0146] In summary, step S104 can be understood as the GPU offsetting the texture coordinates based on the offset sampled over time and the vector field information in the preset flowmap, and then performing texture offset sampling based on the offset coordinates. For example, at the current sampling time t1, the GPU determines the offset a based on the current sampling time t1 and the offset b based on the vector field information in the preset flowmap. Then, the GPU determines the offset texture coordinates based on the offset a and offset b, and performs sampling based on the offset texture coordinates.

[0147] It can be understood that if each sampling moment is executed according to this step, then each sampling moment can obtain the corresponding N1 caustic simulation stickers. Figure 1 .

[0148] S105, GPU will paste N1 caustics simulations corresponding to the current sampling moment Figure 1 The fusion is performed to obtain the caustic frame image corresponding to the current sampling moment (referred to as caustic frame image 1 or the first frame image).

[0149] For N1 caustics simulation stickers Figure 1 Fusion can be understood as applying N1 caustic simulations to Figure 1 Superposition, that is, N1 caustic simulation stickers Figure 1 The texture values ​​corresponding to each texture coordinate in are added together.

[0150] It should be understood that for N1 caustics simulation stickers Figure 1 Using different overlay methods, the resulting caustic frame image can have different caustic texture clarity, depth, and density (i.e., caustic granularity) in various ways. These overlay methods can include using different numbers of textures for overlay and / or using different weights for each texture during overlay.

[0151] For example, Figure 14 This is a schematic diagram of different caustic frame images obtained by using different superposition methods provided in an embodiment of the present application. It can be seen that Figure 14 In Figure (a), the caustic texture is blurry, shallow, and sparse (the caustic granularity is coarser). Figure 14 Compared with Figure (b) Figure 14 In (a), the caustic texture is clearer and deeper, but the caustic texture is sparser (the caustic granularity is coarser). Figure 14 The (c) image has the clearest caustic texture, the deepest caustic texture, and the densest caustic texture (the caustic granularity is the finest).

[0152] In this embodiment, by fusing multiple caustic simulation maps, the caustic effect simulated by the caustic frame image is obtained to be more realistic and natural, thereby improving the user's visual experience.

[0153] S106 : The GPU stores the caustic frame image 1 in a frame buffer.

[0154] Of course, while rendering the caustic frame image 1, the GPU also renders other view controls and stores the rendered data (called other rendering output data) in the frame buffer, which will not be described in detail here.

[0155] S107 , a surface flinger obtains the caustic frame image 1 from the frame buffer and performs layer synthesis to obtain an interface to be displayed.

[0156] Specifically, the layer compositor obtains the caustic frame image and other rendering output data from the frame buffer to perform layer synthesis to obtain the interface to be displayed.

[0157] S108: The layer synthesizer sends the interface to be displayed to the display.

[0158] S109: The display shows the interface to be displayed.

[0159] The above describes the process of the graphics rendering module and GPU rendering the caustic frame image 1 after the lock screen application sends the rendering instruction, and the process of displaying the interface including the caustic frame image 1. It can be understood that at the next sampling moment, returning to execute the above steps S104 to S109, a frame of caustic frame image can be rendered at each sampling moment, and then the interface is displayed. In the above rendering process, the caustic frame image at each sampling moment is generated by N1 caustic simulation stickers. Figure 1 Superposition, and each caustic simulation sticker Figure 1 The noise caustic seamless map is obtained by sampling the texture offset over time. Therefore, the caustic simulation map obtained at each sampling moment changes with time. Figure 1 Different, so the fused caustic frame images are different. In this way, the dynamic change effect of caustics can be simulated.

[0160] The above steps S104 to S109 are also referred to as the first processing process. Each sampling moment in the first processing process is also referred to as the first sampling moment, and any one of the first sampling moments is also referred to as the first target moment.

[0161] It should be noted that the electronic device needs to refresh the interface according to the preset refresh rate. Therefore, after the duration of the refresh cycle (i.e., 1 / refresh rate), the lock screen application can send a refresh instruction to the graphics rendering module. The refresh instruction is used to instruct the refresh interface, that is, to re-render the interface to be displayed. In the embodiment of the present application, when the lock screen application determines that the currently displayed interface includes a caustic dynamic effect and does not detect the user's touch operation on the current interface, it may not send a refresh instruction to the graphics rendering module. In this way, the electronic device continues to execute steps S104 to S109, thereby continuously rendering the caustic frame image, and then presenting the caustic dynamic effect.

[0162] As an optional method, when the user does not switch the interface to another interface, the electronic device can continue to execute the above steps S104 to S109, so that the caustic dynamic effect continues to be displayed, that is, the caustic continues to be in a shaking state.

[0163] Alternatively, a caustic lifecycle can be preset, with the start time of the caustic lifecycle being the display time of the first caustic frame image 1. When the caustic lifecycle ends, steps S104 to S105 are stopped, and a fixed caustic frame image (e.g., the last caustic frame image in the caustic lifecycle) is used for layer compositing. This creates a caustic effect that persists for a period of time before stabilizing. This approach can also save power consumption in electronic devices.

[0164] The beneficial effects achieved by the method provided in this embodiment are summarized and described below.

[0165] 1) The method provided in this embodiment performs texture sampling over time based on a noise caustic seamless map and a preset flowmap, simulating the time-varying process of caustics to form a dynamic caustic effect. The entire process does not require the production of caustic animation or a complex mesh model, thus saving the performance overhead of electronic devices and improving the operating efficiency of electronic devices.

[0166] 2) In the method provided in this embodiment, the noise caustic seamless map can be generated offline without the need for real-time generation, which further saves the performance overhead of the electronic device and further improves the operating efficiency of the electronic device.

[0167] 3) A noise caustic seamless map is an image generated using a noise algorithm and / or noise data. It exhibits randomness and is relatively realistic and natural. Furthermore, a noise caustic seamless map is seamless, with no noticeable seams or discontinuities between textures. Therefore, in the method provided in this embodiment, texture offset sampling based on the noise caustic seamless map produces a realistic and natural caustic simulation image without noticeable seams or discontinuities, resulting in a superior image quality. This, in turn, ensures that the subsequently generated caustic frames are realistic and natural, resulting in a superior caustic dynamic effect and an improved user experience.

[0168] 4) The method provided in this embodiment disturbs the texture sampling of the noise caustic seamless map by presetting the flowmap, so that the obtained caustic simulation map can exhibit the characteristics of quantitative flow, further increase the randomness of the caustic change over time, and make the simulated caustic change process more realistic and natural, thereby making the subsequent caustic dynamic effect better and improving the user experience.

[0169] The following further explains the process of sampling texture offset over time based on noise caustic seamless mapping and the perturbation of sampling through a preset flowmap.

[0170] It can be understood that each texture coordinate in the preset flowmap corresponds to each texture coordinate in the caustic simulation map. Based on this, texture offset sampling can be performed on each texture coordinate in the preset flowmap, and the obtained texture value can represent the vector field information corresponding to the texture coordinate. Afterwards, the texture coordinates are offset using the vector field information (i.e., the texture value sampled from the preset flowmap). Then, the noise caustic seamless map is texture offset sampled over time based on the offset texture coordinates. In this way, disturbances in the process of time-shifted sampling can be achieved, resulting in a simulated flow effect. This is further explained below with reference to the accompanying drawings.

[0171] For example, see Figure 15 In the above step "S104, the GPU performs N1 sets of texture offset sampling on the noise caustic seamless map at the current sampling moment, and in the sampling process, performs sampling perturbation through the preset flowmap to obtain N1 caustic simulation maps corresponding to the current sampling moment. Figure 1 ",include:

[0172] S1041. At the current sampling moment, the GPU performs texture offset sampling on each texture coordinate in the preset flowmap over time to obtain an offset 1 (also called a first offset) corresponding to each texture coordinate.

[0173] In other words, texture offset sampling is performed on each texture coordinate in the preset flowmap based on the current sampling moment to obtain the offset 1 corresponding to each texture coordinate.

[0174] Specifically, the offset 1 corresponding to any texture coordinate UV is expressed as uv_offset, and the dynamic texture offset sampling of the texture coordinate UV over time can be expressed as formula (1):

[0175] uv_offset=texture(flowmap,UV+1.0*t).x-0.5(1)

[0176] Wherein, texture() represents a texture sampling function, flowmap represents sampling a texture value from a flowmap, and in this embodiment, flowmap is a preset flowmap.

[0177] UV represents the texture coordinates, and t represents the coordinate value corresponding to the sampling moment. Optionally, the sampling moment can be converted into a coordinate value through a preset conversion method to facilitate calculation with the texture coordinates. UV+1.0*t represents the texture coordinates of the sampled texture value (i.e., the sampling position). .x represents the x-axis component of the texture coordinates. It can be understood that the flowmap is a grayscale image, and the offsets of the R and G channels are equal. Therefore, the offsets obtained by sampling the x-axis component and the y-axis component of the flowmap are equal, so the x-axis component can be used for calculation, which can simplify the calculation process. 0.5 is an adjustment parameter. As mentioned above, the range of the texture coordinate value is [0,1]. By adjusting the parameter 0.5, the value of uv_offset can be adjusted to the range of [-0.5,0.5]. It can be understood that in other embodiments, 0.5 can also be replaced by other values.

[0178] In general, formula (1) indicates that the texture value is sampled from the texture coordinate UV+1.0*t in the flowmap, and the texture value is obtained by adjusting it by 0.5, and the texture value is used as the offset 1uv_offset corresponding to the texture coordinate UV.

[0179] It can be understood that in formula (1), the texture coordinates for obtaining the texture value are UV+1.0*t. Therefore, as time changes, the texture coordinates for obtaining the texture value from the flowmap change, and so the obtained uv_offset changes. In this way, the texture offset sampling of the flowmap over time is achieved.

[0180] S1042. The GPU offsets each texture coordinate according to the offset 1 corresponding to each texture coordinate to obtain the offset coordinate corresponding to each texture coordinate (referred to as the perturbed coordinate 1 or the first target coordinate).

[0181] Specifically, the perturbed coordinate 1 corresponding to any texture coordinate UV is expressed as caustics_uv, and caustics_uv can be expressed as the following formula (2):

[0182] caustics_uv=UV+uv_offset(2)

[0183] S1043, GPU performs N1 sets of texture offset sampling over time on each disturbed coordinate 1 of the noise caustic seamless map according to the current sampling moment, and obtains N1 caustic simulation maps corresponding to the current sampling moment. Figure 1 .

[0184] As described in the above embodiment, N1 sets of texture offset sampling with different offsets can be performed on the noise caustic seamless map over time to obtain N1 caustic simulation maps. Figure 1. Optionally, different offset multiples can be set to multiply the coordinates corresponding to the sampling moment to obtain different offsets. For example, in N1 groups of texture offset sampling, the offset multiple of the first group of texture offset sampling is 0, and the corresponding offset is t; the offset multiple of the second group of texture sampling is 1, and the corresponding offset is t; the offset multiple of the third group of texture sampling is 2, and the corresponding offset is 2t; and so on, N1 groups of different offsets can be obtained, based on which the noise caustic seamless map is texture offset sampled. The following is an explanation with the formula.

[0185] For example, for any caustics_uv, a first set of texture offset sampling with an offset multiplier of 0 is performed based on the current sampling moment, and the texture value obtained by sampling is expressed as A. The sampling process can be expressed as the following formula (3):

[0186] A=texture(caustics,caustics_uv)(3)

[0187] Where texture() represents the texture sampling function. caustics represents sampling a texture value from the noise caustics seamless map. caustics_uv represents the texture coordinate (i.e., position) of the sampled texture value, caustics_uv, which is the result of formula (2), i.e., the perturbed coordinate 1. Overall, formula (3) represents sampling a texture value from the texture coordinate caustics_uv in the noise caustics seamless map.

[0188] caustics_uv can be understood as caustics_uv + 0 * t, where t represents the texture coordinate at the sampling time. In other words, in this set of samples, based on the current sampling time, a texture offset of 0 is performed with an offset multiplier of 0 and an offset of 0 (i.e., no offset is performed).

[0189] It can be understood that by sampling the perturbed coordinate 1 corresponding to each texture coordinate in the noise caustic seamless map according to formula (3), a caustic simulation map perturbed by flowmap sampling can be obtained, which is recorded as caustic simulation map a.

[0190] For example, for any caustics_uv, a second set of texture offset sampling with an offset multiplier of 1 is performed based on the current sampling moment, and the texture value obtained by sampling is expressed as B. The sampling process can be expressed as the following formula (4):

[0191] B=texture(caustics,caustics_uv+t)(4)

[0192] Where caustics_uv+t indicates that the texture coordinate of the sampled texture value is caustics_uv+t. Similarly, caustics_uv is the result of the calculation of formula (2), that is, the perturbed coordinate 1. Overall, formula (4) represents the texture value sampled from the texture coordinate caustics_uv+t in the noise caustics seamless map.

[0193] caustics_uv+t can be understood as caustics_uv+1.0*t, where t represents the texture coordinate corresponding to the current sampling time. In other words, in this set of samples, based on the current sampling time, a texture offset sampling with an offset multiplier of 1.0 and an offset of 1.0*t is performed.

[0194] It can be understood that by sampling the perturbed coordinate 1 corresponding to each texture coordinate in the noise caustic seamless map according to formula (4), a caustic simulation map that has been perturbed by flowmap sampling and has an offset of 1.0*t over time can be obtained, which is recorded as the caustic simulation map b.

[0195] For example, for any caustics_uv, a third set of texture offset sampling with an offset multiple of 2 is performed based on the current sampling moment, and the texture value obtained by sampling is expressed as C. The sampling process can be expressed as the following formula (5):

[0196] C=texture(caustics,caustics_uv+2.0*t)(5)

[0197] Here, caustics_uv+2.0*t indicates that the texture coordinates of the sampled texture value are caustics_uv+2.0*t. Similarly, caustics_uv is the result of the calculation of formula (2), that is, the perturbed coordinate 1. Overall, formula (5) indicates that the texture value is sampled from the texture coordinate caustics_uv+2.0*t in the noise caustics seamless map. In other words, in this set of samples, based on the current sampling time, the texture offset sampling is performed with an offset multiplier of 2.0 and an offset of 2.0*t.

[0198] It can be understood that by sampling the perturbed coordinate 1 corresponding to each texture coordinate in the noise caustic seamless map according to formula (5), a caustic simulation map that has been perturbed by flowmap sampling and has an offset of 2.0*t over time can be obtained, which is recorded as the caustic simulation map c.

[0199] The subsequent sampling of other groups of texture offsets based on the current sampling moment is similar and will not be repeated here. In this way, multiple caustic simulation maps can be obtained. Among them, the number of caustic simulation maps can be selected according to needs, that is, the number of N1 can be selected according to needs. For example, in the current scene, the number of caustic simulation maps can be selected as 2 (that is, N1 = 2), that is, according to the above formulas (3) and (4), caustic simulation map a and caustic simulation map b can be generated respectively.

[0200] Please continue to see Figure 15 Based on the N1 caustics simulation maps obtained according to the above process, the above step "S105, GPU fuses the N1 caustics simulation maps corresponding to the current sampling moment to obtain caustics frame image 1" includes:

[0201] S1051, GPU will be the current sampling time corresponding to the N1 caustic simulation paste Figure 1 The texture values ​​of each texture coordinate in are weighted summed to obtain the caustic frame image 1.

[0202] Specifically, the texture value of any texture coordinate UV in the caustic frame image 1 is recorded as final. Taking N1=2 as an example, final can be calculated by the following formula (6):

[0203] final=A+weight*B (6)

[0204] Among them, weight represents the weight corresponding to B. The specific value of weight can be set according to requirements.

[0205] It can be understood that the caustic frame image 1 can be obtained by calculating the texture value of each disturbed coordinate 1 in the noise caustic seamless map according to formula (6).

[0206] In this embodiment, a preset flowmap is sampled over time using a texture offset to obtain different offsets 1 at different times, thereby obtaining different vector field information. This sampling perturbation is performed using different vector field information at different times, further increasing the randomness of the caustics' temporal variations. This makes the simulated caustics' evolution more realistic and natural, resulting in better caustic dynamics and an enhanced user experience.

[0207] Step 2: Perform preset operations on the lock screen containing the Caustics live wallpaper.

[0208] As mentioned above, caustics will change with the fluctuation of the medium or light. For example, the caustics on the water surface will shake with the fluctuation of the water surface. In an embodiment of the present application, when the lock screen wallpaper is a caustic dynamic wallpaper, the user can perform a preset operation on the lock screen interface. The electronic device responds to the preset operation and simulates the movement of the caustics according to the user's operation to increase the fun of the interaction between the electronic device and the user and improve the user experience. The following is an explanation with reference to the accompanying drawings. In addition, in this embodiment, different caustic effects can be presented in different areas of the interface according to the user's touch trajectory, further improving the authenticity of the scene simulation and improving the user experience.

[0209] For example, Figure 16 This is another example of interface change diagram provided in the embodiment of the present application. Figure 16 As shown in Figure (a), the phone is in the lock screen state, and the lock screen interface 601 is displayed on the screen. The lock screen wallpaper in the lock screen interface 601 is a dynamic effect of water surface caustics, and the caustics are in a static state. In response to the user performing a preset operation in the lock screen interface 601, the caustics effect in the interface changes, for example, the density of the caustics, the depth of the caustics, etc., and the closer the position is to the user's touch position, the more obvious the caustics change, and the farther the position is from the user's touch position, the less obvious the caustics change, as shown in FIG. Figure 16 As shown in the interface 1601 in (b) of FIG. Optionally, the preset operation can be a long press operation, a sliding operation or a click operation, etc., which is not limited in this application. Figure 16 The sliding operation is used as an example for explanation.

[0210] The image processing method provided in the embodiment of the present application is described below in conjunction with the above-mentioned interface change process.

[0211] Figure 17 : is a flowchart of another image processing method provided in an embodiment of the present application, the method comprising:

[0212] S201. The lock screen application sends a refresh instruction to the graphics rendering module in response to a preset operation performed by the user on the lock screen interface. The refresh instruction carries a touch track corresponding to the preset operation.

[0213] The refresh command is used to instruct the interface to be refreshed according to the touch track, that is, to re-render the interface.

[0214] S202 : The graphics rendering module generates flowmap1 according to the touch track in response to the refresh instruction.

[0215] Specifically, the graphics rendering module can record the vector field information corresponding to the touch track to the corresponding texture coordinates in the flowmap, thereby generating flowmap 1. In this way, flowmap 1 contains the vector field information when the user performs the touch operation.

[0216] S203: The graphics rendering module inputs flowmap1 into the GPU.

[0217] S204, GPU performs N2 sets of texture offset sampling on the noise caustic seamless map at the current sampling moment, and in the sampling process, performs sampling perturbation through flowmap1 to obtain N2 caustic simulation maps corresponding to the current sampling moment. Figure 2 (Also referred to as the second simulation map.) N2 is an integer greater than or equal to 2.

[0218] S205, GPU will be the current sampling time corresponding to the N2 caustic simulation map Figure 2 The fusion is performed to obtain a caustic frame image 2 (also called a second frame image).

[0219] S206 , GPU rendering stores the caustic frame image 2 in a frame buffer.

[0220] S207 , the surface flinger obtains the caustic frame image 2 from the frame buffer and performs layer synthesis to obtain a refreshed interface.

[0221] S208. The layer compositor sends the refreshed interface to the display.

[0222] S209: The display shows the refreshed interface.

[0223] Steps S203 to S209 are similar to the above steps S103 to S109, except that the sampling disturbance is performed through flowmap1 instead of the preset flowmap.

[0224] In this embodiment, when a user performs a preset operation on the lock screen, flowmap1 is generated based on the touch trajectory corresponding to the preset operation. Flowmap1 thus contains vector field information related to the user's touch trajectory. Therefore, the caustic dynamic effect generated based on flowmap1 is related to the user's touch trajectory, creating the illusion of caustics moving along the user's touch trajectory. This improves the realism of the caustic dynamic effect and enhances the fun of the interaction.

[0225] Optionally, in the second step above, a caustic lifecycle can be preset, with the start time of the caustic lifecycle being the display time of the first caustic frame image 2. When the caustic lifecycle ends, steps S204 to S205 are stopped, and a fixed caustic frame image (e.g., the last caustic frame image in the caustic lifecycle) is used for layer synthesis. This creates a caustic effect that persists for a period of time before becoming static. This approach can also save power consumption in electronic devices.

[0226] The above steps S204 to S209 are also referred to as a second processing process, each sampling moment in the second processing process is also referred to as a second sampling moment, and any one of the second sampling moments is also referred to as a second target moment.

[0227] The specific implementation process of the above step S204 can be similar to the process of steps S1041 to S1043, except that flowmap1 is used in this embodiment instead of the preset flowmap. Figure 18 This is a flow chart of another image processing method provided in the embodiment of the present application. Figure 18 As shown in the above step "S204, GPU performs N2 sets of texture offset sampling on the noise caustic seamless map at the current sampling moment, and in the sampling process, performs sampling perturbation through flowmap1 to obtain N2 caustic simulation maps corresponding to the current sampling moment. Figure 2 ",include:

[0228] S2041. At the current sampling moment, the GPU performs texture offset sampling on each texture coordinate in flowmap1 over time to obtain an offset 2 (also called a second offset) corresponding to each texture coordinate.

[0229] It is understood that the calculation method of offset 2 can be the same as that of offset 1. The difference is that offset 1 is obtained based on the preset flowmap texture offset sampling, while offset 2 is obtained based on the flowmap1 texture offset sampling. They are distinguished here by the suffixes 1 and 2. Other names distinguished by suffixes in this application are similar and will not be repeated here.

[0230] S2042. The GPU offsets each texture coordinate according to the offset 2 corresponding to each texture coordinate to obtain a perturbed coordinate 2 (or second target coordinate) corresponding to each texture coordinate.

[0231] S2043, GPU performs N2 sets of texture offset sampling over time on each disturbed coordinate 2 of the noise caustic seamless map according to the current sampling moment, and obtains N2 caustic simulation maps corresponding to the current sampling moment. Figure 2 .

[0232] In this embodiment, the caustics simulation sticker Figure 2 The number of is N2. Optionally, N2 is an integer greater than or equal to 3.

[0233] As an optional implementation, in this embodiment, when fusing caustic simulation maps, the number of caustic simulation maps used can be determined based on the gradient of offset 2 corresponding to each texture coordinate. Regions with different gradients are fused using different numbers of caustic simulation maps to achieve caustic effects of varying densities in different regions, further enhancing the fidelity of the caustic effect. The gradient of offset 2 is also referred to as the first gradient.

[0234] Specifically, after the above step S2041, the method further includes:

[0235] S2044. The GPU calculates the change gradient of the offset 2 corresponding to each texture coordinate within a preset time period.

[0236] The preset time period may be a time period of a preset length before the current sampling moment. In a specific embodiment, the preset time period may be a time period from the previous sampling moment (denoted as t0) to the current sampling moment (denoted as t1). Then, for any texture coordinate UV, the corresponding gradient of the offset 2 can be calculated using formula (7):

[0237]

[0238] in, Indicates the gradient of the texture coordinate UV offset 2 from the previous sampling time t0 to the current sampling time t1. uv_offset(t1) indicates the offset 2 of the texture coordinate UV at the current sampling time t1. uv_offset(t0) indicates the offset 2 of the texture coordinate UV at the previous sampling time t0. t1-t0 indicates the time difference between the previous sampling time t0 and the current sampling time t1.

[0239] It is understandable that Figure 18 In the embodiment, step S2044 is performed between step S2043 and step S2051. In practical applications, step S2044 can also be performed at other times, for example, between step S2041 and step S2042, or between step S2042 and step S2043, etc. This application does not limit this.

[0240] On this basis, the above step "S205, GPU maps the N2 caustic simulation images corresponding to the current sampling moment Figure 2 Performing fusion to obtain the caustic frame image 2" may include:

[0241] S2051. For any texture coordinate UV, the GPU determines whether the change gradient of the offset 2 corresponding to the texture coordinate UV is greater than a preset gradient threshold; if so, execute step S2052; if not, execute step S2053.

[0242] S2052, GPU for N2 caustics simulation Figure 2 The texture values ​​corresponding to the texture coordinates UV in the caustic frame image 2 are weightedly summed to obtain the texture values ​​corresponding to the texture coordinates UV in the caustic frame image 2.

[0243] The texture value of any texture coordinate UV in the caustic frame image 2 is recorded as final. Taking N2=3 as an example, final can be calculated by the following formula (8):

[0244] final=A+weight0*B+weight1*C(8)

[0245] Among them, weight0 represents the weight corresponding to B, and weight1 represents the weight corresponding to C. The specific values ​​of weight0 and weight1 can be set according to needs. A, B, and C represent a caustic simulation map. Figure 2 The texture value corresponding to the texture coordinate UV in .

[0246] S2053, GPU for N3 caustics simulation Figure 2 The texture values ​​corresponding to the texture coordinates UV in the caustic frame image 2 are weightedly summed to obtain the texture values ​​corresponding to the texture coordinates UV in the caustic frame image 2. Wherein, N3 is an integer greater than or equal to 2, and N3 is less than N2.

[0247] Among them, N3 caustic simulation stickers Figure 2 For caustics simulation from N2 sheets Figure 2 There is no limit to the selection method. For example, N3 = 2, N2 = 3, you can select from 3 caustic simulation maps. Figure 2 Select two caustic simulation stickers with offset multiples of 0 and 1 Figure 2 You can also select two caustic simulation stickers with offset multiples of 1 and 2 Figure 2 You can also select two caustic simulation stickers with offset multiples of 0 and 2 Figure 2 The calculation process of this step can be found in the above formula (6), which will not be repeated here.

[0248] That is to say, if the gradient of the offset 2 corresponding to the texture coordinate UV is greater than the preset gradient threshold, a larger number of caustic simulation patches are used. Figure 2 The texture value of the texture coordinate is superimposed. If the gradient of the offset 2 corresponding to the texture coordinate UV is less than or equal to the preset gradient threshold, a smaller number of caustic simulation tiles are used. Figure 2 Adds the texture value at this texture coordinate.

[0249] It can be understood that by executing the above S2044 and S2051 to S2053 for each texture coordinate, the fusion processing of all texture coordinates can be achieved to obtain the caustic frame image 2.

[0250] As mentioned above, flowmap1 contains the vector field information corresponding to the user touch track. In the process of texture offset sampling of flowmap1 over time, the change gradient of the offset sampling result of each texture coordinate (that is, the sampled texture value, that is, uv_offset) can indicate whether the touch track is included.

[0251] Specifically, if the gradient of the change of the offset 2 corresponding to the texture coordinate is large (greater than the preset gradient threshold), it means that the texture coordinate is likely to contain the user's touch track, so the texture values ​​corresponding to the texture coordinate in a larger number (N2 sheets) of caustic simulation maps are superimposed to obtain the texture value corresponding to the texture coordinate. This can make the caustic texture at the texture coordinate denser. If the gradient of the change of the offset 2 corresponding to the texture coordinate is small (less than or equal to the preset gradient threshold), it means that the texture coordinate is likely to not contain the user's touch track, so the texture values ​​corresponding to the texture coordinate in a smaller number (N3 sheets) of caustic simulation maps are superimposed to obtain the texture value corresponding to the texture coordinate. This can make the caustic texture at the texture coordinate sparser. Through the above process, caustic effects of different densities in different areas can be achieved according to the user's touch track, thereby improving the fun and realism of the caustic effect and thus improving the user experience.

[0252] In one embodiment, N1 is less than N2. N2 may or may not be equal to N3. That is, when a user sets a caustics dynamic wallpaper and does not perform a preset operation, a smaller number of caustics simulation maps are selected for overlay. Since the user does not perform a preset operation, there is no need to perform image fusion based on the user's touch trajectory. Directly selecting a smaller number of caustics simulation maps for overlay simplifies algorithm operation efficiency and saves power consumption of electronic devices.

[0253] The above embodiment uses caustics as an example to illustrate the image processing method provided by the embodiment of the present application. It should be noted that the method provided by the present embodiment can also be applied to other scenes, and used to render images with other effects to present other dynamic effects, such as rendering the dynamic effect of waves, the flow effect of waterfalls, the dynamic effect of foam, etc. For the dynamic effect of foam, the method provided by the embodiment of the present application can be used to dynamically superimpose foam maps in the time domain based on flowmap according to the changing gradient of the offset (i.e., the changing gradient of the vector) to achieve foam frame images with different details, thereby presenting a dynamic effect of foam, such as Figure 19 shown.

[0254] The above describes in detail an example of the image processing method provided by the embodiment of the present application. It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0255] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each function can be divided into various functional modules, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0256] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0257] The electronic device provided in this embodiment is used to execute the above-mentioned image processing method, and thus can achieve the same effect as the above-mentioned implementation method.

[0258] When integrated, the electronic device may also include a processing module, a storage module, and a communication module. The processing module may be used to control and manage the operation of the electronic device. The storage module may be used to support the execution of program code and data stored in the electronic device. The communication module may be used to support communication between the electronic device and other devices.

[0259] The processing module may be a processor or a controller. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, or a Wi-Fi chip.

[0260] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a Figure 3 Device with the structure shown.

[0261] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the image processing method of any of the above embodiments.

[0262] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0263] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the image processing method in the above-mentioned method embodiments.

[0264] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0265] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0266] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0267] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0268] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0269] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0270] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image processing method, the method being executed by an electronic device, characterized in that: The method comprises: In response to a first operation of the user, a first flow map and a first texture map are obtained, wherein the first flow map is a texture map including preset vector field information, and the first texture map is a seamless texture map generated based on noise; At a plurality of first sampling moments, respectively executing a first processing process; The first processing process includes: At a first target time, performing N1 sets of time-dependent texture offset sampling on the first texture map, and performing sampling perturbation using the first flow map during the sampling process, to obtain N1 first simulation maps corresponding to the first target time, where N1 is an integer greater than 1, and the first target time is any one of the plurality of first sampling times; Fusing the N1 first simulation maps to obtain a first frame image corresponding to the first target moment; The first frame of image is displayed.

2. The method according to claim 1, characterized in that At the first target moment, performing N1 sets of texture offset sampling over time on the first texture map, and performing sampling perturbation through the first flow map during the sampling process to obtain N1 first simulation maps corresponding to the first target moment, including: At the first target moment, performing texture offset sampling on each texture coordinate in the first flow map over time to obtain a first offset corresponding to each texture coordinate; offsetting each texture coordinate according to a first offset corresponding to each texture coordinate to obtain a first target coordinate corresponding to each texture coordinate; According to the first target moment, N1 groups of texture offset sampling over time are performed on each first target coordinate in the first texture map to obtain the N1 first simulation maps.

3. The method according to claim 2, characterized in that The fusing the N1 first simulation maps to obtain a first frame image corresponding to the first target moment includes: The texture values ​​of each of the target coordinates in the N1 first simulation maps are weightedly summed to obtain the first frame image.

4. The method according to any one of claims 1 to 3, characterized in that The first processing process stops executing after executing a preset time period.

5. The method according to any one of claims 1 to 4, characterized in that The time interval between two adjacent first sampling moments is equal to the refresh period corresponding to the refresh rate of the electronic device.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: receiving a touch operation performed by a user on a first interface, where the first interface includes the first frame of image; In response to the touch operation, obtaining a touch track corresponding to the touch operation; generating a second flow map according to the touch trajectory, where the second flow map is a texture map including vector field information corresponding to the touch trajectory; At a plurality of second sampling moments, respectively executing a second processing process; The second processing process includes: At a second target time, performing N2 sets of texture offset sampling over time on the first texture map, and performing sampling perturbation using the second flow map during the sampling process to obtain N2 second simulation maps corresponding to the second target time, where N2 is an integer greater than 1, and the second target time is any one of the plurality of second sampling times; Fusing the N2 second simulation maps to obtain a second frame image corresponding to the second target moment; The second frame image is displayed.

7. The method according to claim 6, characterized in that At the second target moment, performing N2 sets of texture offset sampling on the first texture map over time, and performing sampling perturbation through the second flow map during the sampling process to obtain N2 second simulation maps corresponding to the second target moment, including: At the second target moment, performing texture offset sampling on each texture coordinate in the second flow map over time to obtain a second offset corresponding to each texture coordinate; offsetting each texture coordinate according to the second offset corresponding to each texture coordinate to obtain a second target coordinate corresponding to each texture coordinate; According to the second target moment, N2 groups of texture offset sampling over time are performed on each second target coordinate in the first texture map to obtain the N2 second simulation maps.

8. The method according to claim 7, characterized in that After performing texture offset sampling on each texture coordinate in the second flow map over time at the second target moment to obtain a second offset corresponding to each texture coordinate, the method further includes: A change gradient of a second offset corresponding to a first texture coordinate within a preset time period before the first sampling moment is determined to obtain a first change gradient, where the first texture coordinate is any texture coordinate in the second flow map.

9. The method according to claim 8, characterized in that The preset time period is a time period between a sampling moment before the first sampling moment and the first sampling moment.

10. The method according to claim 8 or 9, characterized in that N2 is an integer greater than 2, and fusing the N2 second simulation maps to obtain a second frame image corresponding to the second target moment includes: If the first change gradient is greater than a preset gradient threshold, performing weighted summation on the texture values ​​corresponding to the first texture coordinate in the N2 second simulated maps to obtain the texture value corresponding to the first texture coordinate in the second frame image; If the first change gradient is less than or equal to the preset gradient threshold, a weighted sum is performed on the texture values ​​corresponding to the first texture coordinates in N3 second simulation maps to obtain the texture value corresponding to the first texture coordinates in the second frame image, where N3 is an integer greater than 1 and less than N2.

11. The method according to claim 10, characterized in that N1 is smaller than N2.

12. The method according to any one of claims 6 to 11, characterized in that The first interface is a lock screen interface or a desktop.

13. The method according to any one of claims 1 to 12, characterized in that The obtaining of the first flow map and the first texture map includes: The first flow map and the first texture map are obtained from a server or a memory of the electronic device.

14. The method according to any one of claims 1 to 13, characterized in that The first operation is an operation of setting the first wallpaper as a lock screen wallpaper and / or desktop wallpaper, and the first wallpaper is a dynamic wallpaper corresponding to the first texture map.

15. The method according to any one of claims 1 to 14, characterized in that The first texture map has a caustic effect.

16. An electronic device, characterized in that: The electronic device includes: one or more processors, and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 15.

17. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises instructions, which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 15.

Images