Image processing method and electronic device

Abstract

Embodiments of the present application provide an image processing method and an electronic device. The method comprises: in response to a first operation of a user, acquiring a first flow map and a first texture map, the first flow map being a texture map containing preset vector field information, and the first texture map being a seamless texture map generated according to noise; at a plurality of first sampling time points, respectively performing a first processing process; the first processing process comprises: at a first target time point, performing N1 sets of texture offset sampling over time on the first texture map, and performing sampling disturbance through the first flow map in the sampling process to obtain N1 simulated maps corresponding to the first target time point, N1 being an integer greater than 1, and the first target time point being any one of the plurality of first sampling time points; fusing the N1 simulated maps to obtain a first frame of image corresponding to the first target time point; and displaying the first frame of image. The method can reduce the performance overhead of the electronic device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronics, in particular to an image processing method and an electronic device. BACKGROUND

[0002] In the application process of electronic devices, in many scenarios, dynamic effects such as caustics and foam are used. These dynamic effects can increase the liveliness of the interface and improve the visual experience of users.

[0003] Taking caustics as an example, in the traditional technology, the display of caustic dynamic effects is generally realized by using a complex mesh model, and the performance overhead of the electronic device is relatively large. SUMMARY

[0004] The present application provides an image processing method and an electronic device, which can reduce performance overhead.

[0005] In a first aspect, the present application provides an image processing method, which is executed by an electronic device, and the method comprises: in response to a first operation of a user, acquiring a first flow direction map and a first texture map, the first flow direction map being a texture map containing preset vector field information, and the first texture map being a seamless texture map generated according to noise; and respectively executing a first processing process at a plurality of first sampling time points.

[0006] The first processing process comprises: at a first target time point, performing N1 sets of texture offset sampling with time on the first texture map, and performing sampling disturbance by using the first flow direction map in the sampling process, to obtain N1 simulated maps corresponding to the first target time point, N1 being an integer greater than 1, and the first target time point being any one of the plurality of first sampling time points; fusing the N1 simulated maps to obtain a first frame image corresponding to the first target time point; and displaying the first frame image.

[0007] It can be understood that the first processing process is respectively executed at the plurality of first sampling time points, that is, the first processing process is repeatedly executed for each first sampling process. Each execution of the first processing process generates and displays a first frame image. The first processing process is a process of performing texture offset sampling with time based on the first texture map (that is, a process of rendering the first frame image), and the texture details of the first frame image obtained by executing the first processing process are different at different first sampling time points. In this way, each first frame image is displayed frame by frame, simulating the change process of the texture in the first texture map with time, and the dynamic effect corresponding to the first texture map can be realized.

[0008] Optionally, the first texture map can be a picture with a preset display effect, such as a picture with a focus effect, a picture with a foam effect, or a picture with a wave effect. Taking the first texture map with a foam effect as an example, the first frame of images also has a foam effect, and the texture details of the first frame of images obtained at different sampling moments are different. In this way, the dynamic effect of the foam can be realized by sequentially displaying a plurality of first frame of images to simulate the change process of the foam over time.

[0009] Optionally, the texture map generated according to the noise refers to a texture map generated by a noise algorithm and / or noise data.

[0010] The image processing method provided in the present application can present a dynamic picture with a display effect corresponding to the first texture map based on the first texture map, and the entire process does not require animation production or complex mesh models, thereby saving the performance overhead of the electronic device and improving the operating efficiency of the electronic device. In addition, the first texture map is a map generated according to noise, and has randomness and is relatively realistic and natural. Moreover, the first texture map is a seamless texture map, and there is no obvious joint or discontinuity between textures. Therefore, in the method provided in the embodiments of the present application, the first simulation picture obtained by performing texture offset sampling on the first texture map based on the first texture map is relatively realistic and natural, and there is no obvious joint or discontinuity, the image effect is good, and the subsequent generated first frame of images is realistic and natural, the dynamic picture effect is good, and the user experience is improved. In addition, in the method provided in the embodiments of the present application, the texture sampling of the first texture map is disturbed by the first flow direction map, the first flow direction map includes preset vector field information, the first simulation picture obtained can exhibit the characteristics of quantitative flow, further increase the randomness of the texture in the first texture map changing over time, make the simulated texture change process more realistic and natural, and thus the subsequent dynamic picture effect is better, and the user experience is improved.

[0011] In a possible implementation manner, at the first target moment, N1 groups of texture offset sampling over time are performed on the first texture map, and sampling disturbance is performed by the first flow direction map during the sampling process to obtain N1 first simulation pictures corresponding to the first target moment, including: at the first target moment, texture offset sampling over time is performed on each texture coordinate in the first flow direction map to obtain a first offset amount corresponding to each texture coordinate; each texture coordinate is offset according to the first offset amount corresponding to each texture coordinate to obtain a first target coordinate corresponding to each texture coordinate; and N1 groups of texture offset sampling over time are performed on each first target coordinate in the first texture map according to the first target moment to obtain N1 first simulation pictures.

[0012] In this implementation, the first flow direction map is sampled by time-based texture offsetting to obtain different first offset values at different time points, that is, different vector field information. In this way, different vector field information is used for sampling disturbance at different time points, thereby further increasing the randomness of the texture change over time in the first texture map, making the simulated texture change process more realistic and natural, and further making the subsequent dynamic picture effect better and improving the user experience.

[0013] In a possible implementation, the N1 first simulation maps are fused to obtain a first frame image corresponding to a first target time, including: performing weighted summation on the texture values of each target coordinate in the N1 first simulation maps to obtain the first frame image.

[0014] In a possible implementation, the first processing process stops after being executed for a preset time length. In this way, the dynamic effect (for example, the dynamic effect of the bokeh) of the texture in the picture is static after lasting for a period of time, thereby saving the power consumption of the electronic device.

[0015] In a possible implementation, the time interval between two adjacent first sampling time points is equal to a refresh period corresponding to the refresh rate of the electronic device.

[0016] That is, the first processing process is executed once every refresh period to obtain a first frame image. In this way, the refresh rate of the electronic device is consistent, which can improve the picture display effect and save power consumption.

[0017] In a possible implementation, the method further includes: receiving a touch operation of a user on a first interface, the first interface including the first frame image; in response to the touch operation, obtaining a touch track corresponding to the touch operation; generating a second flow direction map according to the touch track, the second flow direction map being a texture map including vector field information corresponding to the touch track; and executing a second processing process at a plurality of second sampling time points.

[0018] The second processing process includes: at a second target time, performing N2 sets of time-based texture offset sampling on the first texture map, and performing sampling disturbance by using the second flow direction map in the sampling process to obtain N2 second simulation maps corresponding to the second target time, N2 being an integer greater than 1, and the second target time being any one of the plurality of second sampling time points; fusing the N2 second simulation maps to obtain a second frame image corresponding to the second target time; and displaying the second frame image.

[0019] The second processing process is similar to the first processing process, except that the second processing process is texture offset sampling based on a second flow field map. The second flow field map is a texture map generated according to the touch trajectory and contains vector field information corresponding to the touch trajectory. Therefore, the flow direction of the texture in the second frame image generated based on the second flow field map is related to the touch trajectory of the user, which improves the realism of the dynamic effect and the interest of the interaction.

[0020] In a possible implementation, at the second target time, N2 groups of texture offset sampling are performed on the first texture map over time, and sampling disturbance is performed on the second flow field map during the sampling process to obtain N2 second simulation maps corresponding to the second target time, including: at the second target time, texture offset sampling is performed on each texture coordinate in the second flow field map over time to obtain a second offset corresponding to each texture coordinate; the second offset corresponding to each texture coordinate is used to offset each texture coordinate respectively to obtain a second target coordinate corresponding to each texture coordinate; and N2 groups of texture offset sampling are performed on each second target coordinate in the first texture map over time according to the second target time to obtain N2 second simulation maps.

[0021] The implementation has similar beneficial effects to those of generating the first simulation map, and thus repeated description is omitted.

[0022] In a possible implementation, after the second offset corresponding to each texture coordinate in the second flow field map is obtained by performing texture offset sampling on each texture coordinate in the second flow field map over time at the second target time, the method further includes: determining a first change gradient of the second offset corresponding to the first texture coordinate in a preset time period before the first sampling time to obtain a first change gradient, the first texture coordinate being any texture coordinate in the second flow field map.

[0023] In a possible implementation, the preset time period is a time period between a previous sampling time of the first sampling time and the first sampling time.

[0024] In a possible implementation, N2 is an integer greater than 2, and the second frame image corresponding to the second target time is obtained by fusing the N2 second simulation maps, including: if the first change gradient is greater than a preset gradient threshold, weighted summation is performed on texture values corresponding to the first texture coordinate in the N2 second simulation maps to obtain texture values corresponding to the first texture coordinate in the second frame image; and if the first change gradient is less than or equal to the preset gradient threshold, weighted summation is performed on texture values corresponding to the first texture coordinate in the N3 second simulation maps to obtain texture values corresponding to the first texture coordinate in the second frame image, N3 being an integer greater than 1 and less than N2.

[0025] In the foregoing implementations, if the first change gradient corresponding to the first texture coordinate is large (larger than a preset gradient threshold), it is indicated that the first texture coordinate is likely to contain the touch trajectory of the user, and thus a large number (N2) of texture values corresponding to the first texture coordinate in the second simulation map are superimposed to obtain the texture value corresponding to the first texture coordinate. In this way, the texture at the first texture coordinate can be made denser. If the first change gradient corresponding to the first texture coordinate is small (smaller than or equal to the preset gradient threshold), it is indicated that the first texture coordinate is not likely to contain the touch trajectory of the user, and thus a small number (N3) of texture values corresponding to the first texture coordinate in the second simulation map are superimposed to obtain the texture value corresponding to the first texture coordinate. In this way, the texture at the first texture coordinate can be made sparser. Through the foregoing process, the texture effect with different densities in different regions can be achieved according to the touch trajectory of the user, the interestingness and the realism of the dynamic effect are improved, and the user experience is improved.

[0026] In a possible implementation, N1 is smaller than N2.

[0027] That is, in the first processing process, a small number of first simulation maps are selected for superimposition. Since the scene corresponding to the first process is a scene in which the user does not perform a touch operation in the first interface, image fusion according to the touch trajectory of the user is not needed, and a small number of first simulation maps are directly selected for superimposition, which can simplify the algorithm running efficiency and save the power consumption of the electronic device.

[0028] In a possible implementation, the first interface is a lock screen interface or a desktop.

[0029] In a possible implementation, the first flow direction map and the first texture map are obtained by obtaining the first flow direction map and the first texture map from a server or a storage of the electronic device.

[0030] That is, the first flow direction map and the first texture map can be generated offline, without being generated in real time, which further saves the performance overhead of the electronic device and further improves the running efficiency of the electronic device.

[0031] In a possible implementation, the first operation is an operation of setting the first wallpaper as a lock screen wallpaper and / or a desktop wallpaper, and the first wallpaper is a dynamic wallpaper corresponding to the first texture map.

[0032] In a possible implementation, the first texture map has a caustic effect.

[0033] In a second aspect, the present application provides an apparatus, which is included in an electronic device, and the apparatus has functions to implement the above-mentioned first aspect and the behaviors of the electronic device in the possible implementation manners of the above-mentioned first aspect. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a receiving module or unit, a processing module or unit, and the like.

[0034] In a third aspect, the present application provides an electronic device, which includes one or more processors, and a memory; the memory is coupled to the one or more processors, and the memory is configured to store computer program codes, the computer program codes include computer instructions, and the one or more processors invoke the computer instructions to cause the electronic device to perform any one of the methods in the technical solutions of the first aspect.

[0035] In a fourth aspect, the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are configured to invoke computer instructions to cause the electronic device to perform the method in the first aspect and any possible implementation manner thereof.

[0036] Optionally, the chip system further includes a memory, and the memory is connected to the processor through a circuit or a wire.

[0037] Further optionally, the chip system further includes a communication interface.

[0038] In a fifth aspect, the present application provides a computer readable storage medium, which includes instructions, and when the instructions are run on an electronic device, the electronic device is caused to perform any one of the methods in the technical solutions of the first aspect.

[0039] In a sixth aspect, the present application provides a computer program product, which includes computer program codes, and when the computer program codes are run on an electronic device, the electronic device is caused to perform any one of the methods in the technical solutions of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 FIG. 1 is a schematic diagram of a caustic effect provided by an embodiment of the present application;

[0041] Figure 2 FIG. 2 is a simulation schematic diagram of a water surface caustic effect provided by an embodiment of the present application;

[0042] Figure 3 FIG. 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0043] Figure 4 FIG. 4 is a software structure block diagram of an electronic device provided by an embodiment of the present application;

[0044] Figure 5 is a schematic diagram of interface change provided by an embodiment of the present application;

[0045] Figure 6 is a schematic diagram of a lock screen interface provided by an embodiment of the present application;

[0046] Figure 7 is another schematic diagram of interface change provided by an embodiment of the present application;

[0047] Figure 8 is a schematic diagram of a flow of an image processing method provided by an embodiment of the present application;

[0048] Figure 9 is a schematic diagram of a noise caustics seamless map provided by an embodiment of the present application;

[0049] Figure 10 is a schematic diagram of repeating splicing of a noise caustics seamless map provided by an embodiment of the present application;

[0050] Figure 11 is a schematic diagram of flowmap and vector field information provided by an embodiment of the present application;

[0051] Figure 12 is a schematic diagram of a caustics simulation map provided by an embodiment of the present application;

[0052] Figure 13 is another schematic diagram of a caustics simulation map provided by an embodiment of the present application;

[0053] Figure 14 is a schematic diagram of different caustics frame images obtained by using different superimposition manners provided by an embodiment of the present application;

[0054] Figure 15 is another schematic diagram of a flow of an image processing method provided by an embodiment of the present application;

[0055] Figure 16 is another schematic diagram of interface change provided by an embodiment of the present application;

[0056] Figure 17 is another schematic diagram of a flow of an image processing method provided by an embodiment of the present application;

[0057] Figure 18 is another schematic diagram of a flow of an image processing method provided by an embodiment of the present application;

[0058] Figure 19 is a schematic diagram of a foam dynamic effect provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; in this document, "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0060] Hereinafter, the terms "first", "second", "third" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", "third" can explicitly or implicitly include one or more of the features.

[0061] In the present application, the reference "one embodiment" or "some embodiments" means that in one or more embodiments of the present application, the specific features, structures or characteristics described in connection with the embodiment are included. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in the present application are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0062] In order to better understand the embodiments of the present application, the following explains the terms or concepts that may be involved in the embodiments.

[0063] 1, caustics

[0064] Caustics refers to the bright spot or light and shadow effect formed on other surfaces after the refraction or reflection of light through certain media (such as water surface, glass, etc.). Among them, when the medium is water surface, the caustics formed can be called water surface caustics; when the medium is glass, the caustics formed can be called glass caustics. Exemplary, Figure 1 A schematic diagram of an example caustic effect provided by the embodiments of the present application.

[0065] Caustics will sway with the change of medium or light, for example, water surface caustics will sway with the fluctuation of water surface. That is, caustics can show dynamic effects.

[0066] 2, seamless mapping

[0067] Seamless mapping refers to the phenomenon that no obvious joint or discontinuity appears between textures when the mapping is tiled (repeatedly spliced) in each direction (such as the horizontal direction and the vertical direction).

[0068] 3、Flow map

[0069] A flow map is a kind of texture data recording 2D vector field information. Assuming that there is a 2D plane, each point on the plane corresponds to a vector, which points to the direction in which the point will move next. The information of the vector is recorded by a texture value, and thus a flow map is obtained. The texture value can include a color value. The color value in the flow map usually includes two channels of red (R) and green (G).

[0070] 4、Texture, texture data and texture value

[0071] Texture refers to the details and characteristics of the surface of an object, which is composed of texture primitives and their arrangement rules. In computer graphics, texture data is a set of data representing the details and characteristics of the surface of an object. The texture data can be an image, a pattern or other forms of data. When the texture data is an image, it is called a texture map. The texture data includes texture values corresponding to each texture coordinate, which can represent the color, brightness, transparency and other information of the texture. By applying the texture value to the surface of the object, the realism and details of the object can be increased.

[0072] 5、Texture coordinate

[0073] In computer graphics, in order to map a texture (which can be a 2D texture, a 1D texture or a 3D texture, etc.) to a graph, a texture coordinate (also called a texture mapping coordinate) needs to be associated with each vertex of the graph. By specifying a texture coordinate for each vertex, it can be indicated which part of the texture map to sample the texture value (such as color) from, and then the texture mapping is realized on other fragments of the graph by performing fragment interpolation on the fragments.

[0074] The texture coordinate can be represented by the coordinate values of the x-axis and the y-axis. In a 2D texture map, the range of the x-axis and the y-axis is 0 to 1. That is, in the texture coordinate, the value range of the x-coordinate and the y-coordinate is both [0, 1].

[0075] The application scenarios of the image processing method provided by the embodiments of the present application and the technical problems faced by the present application will be described below.

[0076] The image processing method provided in the embodiments of the present application can be used to realize display of dynamic effects based on light or water, which include but are not limited to dynamic effects of caustics, dynamic effects of splashes, flowing effects of waterfalls, dynamic effects of foam, and the like. The dynamic effects of caustics can be dynamic effects of water surface caustics or dynamic effects of glass caustics.

[0077] Taking the dynamic effects of water surface caustics as an example, in some scenarios, the application can present a realistic picture to the user through the dynamic effects of caustics, so as to simulate a real scene and improve the user experience. For example, in a certain game application (APP), an interface picture simulating a seaside scene needs to be displayed. In this case, the dynamic effects of caustics can be used to make the picture more beautiful and make the simulated seaside scene more realistic. For another example, in a wallpaper APP, a wallpaper with dynamic effects of caustics (hereinafter referred to as dynamic wallpaper of caustics) is provided to the user. The user can select the dynamic wallpaper of caustics as a lock screen wallpaper or a desktop wallpaper, which can not only improve the user's visual experience but also improve the interest. For example, Figure 2 An example of a simulation diagram of the dynamic effects of water surface caustics provided in the embodiments of the present application is shown in FIG. 1. Specifically, the diagram is a certain frame of image in the dynamic effects of water surface caustics generated by an electronic device. Figure 2 As can be seen, the dynamic effects of water surface caustics can improve the realism of the picture and improve the user's visual experience.

[0078] It can be understood that there can be various implementation manners of the dynamic effects based on light or water, for example, the dynamic effects can be realized by vertex animation, or can be realized by a mesh model, or can be realized by real-time calculation of a noise map. However, since these dynamic effects have many details and change irregularly, and can involve simulation of light propagation, refraction, reflection and the like, the dynamic effects are generally complex, and the performance overhead of the electronic device is large when the dynamic effects are realized by using the above methods.

[0079] The image processing method provided in the embodiments of the present application aims to solve the problem.

[0080] The image processing method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and the like, which can install application programs. The embodiments of the present application do not make any limitation on the specific type of the electronic device.

[0081] Exemplary, Figure 3 is a structural schematic diagram of an electronic device 100 provided by an embodiment of the present application. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, and the like. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0082] It can be understood that the structure illustrated by the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0083] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), and the like. Different processing units can be independent devices, or can be integrated into one or more processors.

[0084] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of instruction fetching and instruction execution.

[0085] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0086] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information. In the embodiments of the present application, the image processing method can be realized by cooperation of the GPU and the application processor.

[0087] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, and N is a positive integer greater than 1.

[0088] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0089] Figure 4is a software structure block diagram of the electronic device 100 of the embodiments of the present application. The layered architecture divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer. The application layer can include a series of application packages.

[0090] As shown in Figure 4 , the application package can include camera, gallery, calendar, call, map, navigation, music, video and other applications.

[0091] In the embodiments of the present application, the application package can also include lock screen, game and other applications.

[0092] The application framework layer provides the application layer with application programming interface (API) and programming framework for the application. The application framework layer includes some pre-defined functions.

[0093] As shown in Figure 4 , the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager and the like.

[0094] The window manager is used to manage the window program. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen.

[0095] The content provider is used to store and obtain data, and make the data accessible to the application. The data can include video, image, audio, dialing and answering phone, browsing history and bookmark, phone book and the like.

[0096] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and the like. The view system can be used to build an application. The display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0097] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including connection, hang up, etc.).

[0098] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files and the like.

[0099] The notification manager enables applications to display notification information in the status bar, which can be used to communicate alert-type messages that can automatically disappear after a brief stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the form of a figure or a scroll bar text in the top status bar of the system, such as a notification of a background running application, and can also be a notification that appears in the form of a dialogue window on the screen. For example, the status bar prompts text information, emits a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0100] In the embodiments of the present application, the application framework layer further includes a graphics rendering module and a layer compositor. The graphics rendering module obtains rendering input data and inputs the rendering input data to the GPU. Optionally, the rendering input data can include texture data, vertex data, index data, material data, lighting data, etc. The vertex data is used to describe the vertex characteristics of an object, and the vertex data can include the position coordinates of the vertex, the color value of the vertex, the normal vector of the vertex, the texture coordinates of the vertex, etc. The index data is used to describe how to form a triangle or other graphics primitive to construct the surface of an object. The material data is used to describe the surface material of an object. The lighting data is used to describe the light source condition of the environment where the object is located, including the type, position, direction, color and intensity of the light source, etc.

[0101] The layer compositor can be, for example, SurfaceFlinger, which is used for layer composition to obtain a frame image.

[0102] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0103] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0104] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform object lifecycle management, stack management, thread management, security and exception management, and garbage collection functions.

[0105] The system library can include a plurality of functional modules. For example: a surface manager (surface manager), media libraries, a three-dimensional graphics processing library (for example: OpenGL ES), a 2D graphics engine (for example: SGL), etc.

[0106] The surface manager is used to manage the display subsystem and provides fusion of 2D and 3D layers for multiple applications.

[0107] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0108] A 3D graphics processing library is a programming interface for graphics rendering. It provides developers with a set of instructions and functions to describe the graphics content and operations to be rendered, so as to realize the functions of GPU 3D graphics drawing, image rendering, compositing and layer processing.

[0109] A 2D graphics engine is a graphics engine for 2D drawing.

[0110] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0111] in addition, Figure 4 The document also illustrates the modules of the hardware layer related to the aforementioned software architecture. For example... Figure 4 As shown, the hardware layer can include a GPU, a display, etc. The graphics rendering module of the application architecture layer can input data to the GPU of the hardware layer, for example, input rendering input data. The display can show images based on frame images provided by the layer compositor.

[0112] For ease of understanding, the following embodiments of this application will be described using the following methods: Figure 3 and Figure 4 Taking the electronic device with the structure shown as an example, and taking the application scenario of a lock screen wallpaper with dynamic water surface caustics effect as an example, the image processing method provided by the embodiment of this application will be specifically described in conjunction with two usage processes in this application scenario.

[0113] Step 1: Set a caustic live wallpaper.

[0114] In one embodiment, a user can set a caustic live wallpaper via a wallpaper settings entry. For example, Figure 5 This is a schematic diagram illustrating an example of interface changes provided in an embodiment of this application. Taking a mobile phone as an example, for instance... Figure 5 As shown in Figure (a), the phone's desktop 501 includes an icon 5011 for a settings app. In response to a user's selection of the settings app icon 5011 (e.g., a click), the phone displays... Figure 5 The settings interface 502 is shown in Figure (b). Settings interface 502 includes multiple phone settings, such as Bluetooth, WLAN, and personal hotspot settings. Among these settings is a "Desktop & Personalization" setting 5021. In response to a user's selection (e.g., click) of the "Desktop & Personalization" setting 5021, the phone can display interface 503, such as...Figure 5 As shown in Figure (c), interface 503 includes several personalized settings such as "Wallpaper" settings 5031, "Theme" settings 5032, "Font" settings, and "Icon" settings. Among these, "Wallpaper" settings 5031 is the entry point for wallpaper settings. Optionally, "Wallpaper" settings 5031 can be presented in the form of cards, which can rotate between previews of the phone's lock screen and desktop. For example, the "Wallpaper" settings 5031 in interface 503 currently displays a preview of the lock screen 5032, which shows the effect of the current lock screen wallpaper.

[0115] When a preview image 5032 is displayed in the "Wallpaper" settings item 5031, in response to the user's selection operation (e.g., click operation) on the "Wallpaper" settings item 5031, the phone displays as follows: Figure 5 Figure (d) shows the lock screen wallpaper editing interface 504. The lock screen wallpaper editing interface 504 includes a "lock screen style" control 5041. In response to the user's selection of the "lock screen style" control 5041, the phone displays... Figure 5 Figure (e) shows the lock screen style display interface 505. The lock screen style display interface 505 includes multiple lock screen style options, including the caustic live wallpaper option 5051.

[0116] In response to the user's selection of the caustic live wallpaper option 5051 (e.g., a click), the electronic device renders the caustic live wallpaper to display. Figure 5 The lock screen wallpaper preview interface 506 is shown in Figure (f). It can be seen that in preview image 5061 of the lock screen wallpaper preview interface 506, the lock screen wallpaper has been changed to a caustic live wallpaper. The lock screen wallpaper preview interface 506 includes an "Application" control 5062. In response to the user's selection operation on the "Application" control 5062, the phone sets the lock screen wallpaper to the caustic live wallpaper. When the user locks the screen, the phone displays as shown... Figure 6 The lock screen interface shown is 601. It should be noted that... Figure 6 In the lock screen interface 601, the lock screen wallpaper presents a static caustic image. However, in reality, the lock screen wallpaper can display a dynamic caustic effect. That is, the position and density of the caustics in the lock screen wallpaper will change at different times, and the entire caustic can appear to be shaking. Optionally, the amplitude of the caustic shaking can decrease over time, and after a period of time, the caustic can stop shaking.

[0117] In another embodiment, users can also set a caustic live wallpaper through the theme settings entry. The theme settings entry is used to set the electronic device's lock screen wallpaper, desktop wallpaper, and always-on display content to the same theme.

[0118] For example,Figure 7 This is another example of interface change provided in an embodiment of this application. For example... Figure 7 As shown in Figure (a), the mobile phone displays interface 503. The display process and content of interface 503 can be found in the above description. Figure 5 The embodiments shown are not described in detail here.

[0119] The "Theme" setting item 5032 in interface 503 is the entry point for theme settings. In response to the user's selection of the "Theme" setting item 5032, the phone displays the theme settings interface 701, as shown below. Figure 7 As shown in Figure (b). The theme settings interface 701 includes a caustic theme option 7011. In response to the user's selection of the caustic theme option 7011, the caustic theme settings interface 702 is entered, as shown in Figure (b). Figure 7 As shown in Figure (c), the caustic theme settings interface 702 displays preview images 7021 of the lock screen wallpaper (i.e., the caustic dynamic lock screen wallpaper) under the caustic theme, and 7022 of the desktop wallpaper under the caustic theme.

[0120] The caustic theme settings interface 702 also includes application control 7023. In response to the user's selection of application control 7023, the electronic device sets the lock screen wallpaper, desktop wallpaper, and always-on display to a caustic dynamic effect. Specifically, after setting the lock screen wallpaper, the phone can display the following effect when the screen is locked: Figure 6 The lock screen interface shown is 601.

[0121] The image processing method provided in the embodiments of this application will be described below in conjunction with the above-described interface change process.

[0122] Figure 8 This is a flowchart illustrating an example image processing method provided in an embodiment of this application. The method includes:

[0123] S101. In response to the user's operation of setting a caustic live wallpaper, the lock screen application sends a rendering instruction to the graphics rendering module. The rendering instruction carries information about the caustic effect image and caustic identifier in the interface to be displayed.

[0124] The steps to set a caustic live wallpaper (also known as the first step) can be as described above. Figure 5 The selection operation of the application control 5062 in the interface 506 shown in Figure (f) can also be the above. Figure 7 The selection operation of the application control 7023 in the focus setting interface 702 shown in Figure (c) is not specifically limited in this application.

[0125] The rendering command is used to instruct the rendering of the interface to be displayed, such as the lock screen interface 601.

[0126] Optionally, the rendering instruction can carry layout information and resource information of the interface to be displayed. The layout information is used to indicate the position and size of a view control included in the interface to be displayed. The view control can be an image or other control. The resource information refers to information of the view control (including the image) included in the interface to be displayed. In this embodiment, the resource information includes information of the caustics effect picture. It can be understood that the electronic device can support the user to select multiple caustics dynamic effects, and the information of the caustics effect picture is used to indicate which caustics dynamic effect is in the interface to be displayed. Optionally, the caustics effect picture can be any one of the caustics static pictures in the process of forming the caustics dynamic effect. The information of the caustics effect picture can be the name, number or ID of the caustics effect picture, which is not limited in this regard.

[0127] In addition, in this embodiment, the rendering instruction also carries a caustics identifier. The caustics identifier is used to indicate that the interface to be displayed includes a caustics dynamic effect.

[0128] In S102, the graphics rendering module determines, in response to the rendering instruction, that the caustics identifier exists in the rendering instruction, and acquires a noise caustics seamless map (also referred to as a first texture map) and a preset flowmap (also referred to as a first flow direction map) corresponding to the information of the caustics effect picture.

[0129] Specifically, after the graphics rendering module receives the rendering instruction, it determines, according to the caustics identifier, that the interface to be displayed includes a caustics dynamic effect, and thus further acquires the corresponding noise caustics seamless map according to the information of the caustics effect picture. The noise caustics seamless map is used to render a caustics frame image.

[0130] The noise caustics seamless map is described as follows:

[0131] a. The noise caustics seamless map is a texture map with a caustics effect. The noise caustics seamless map can be a 2D picture.

[0132] b. The noise caustics seamless map is an image generated by using a noise algorithm and / or noise data. It should be understood that in the field of graphics, noise is not a common interference or useless signal, but random data with certain characteristics. These noise data can be processed and utilized by specific algorithms and rules to generate natural, realistic and random images. For example, a noise algorithm and / or noise data can be used to generate a landscape feature map such as mountains, hills and valleys; for another example, a noise algorithm and / or noise data can be used to generate a ripple pattern; for yet another example, a noise algorithm and / or noise data can be used to generate a caustics pattern.

[0133] c. The noise caustics seamless map is a seamless map, that is, when tiling (repeatedly splicing) in all directions, there will be no obvious joints or discontinuous phenomena between textures.

[0134] An example of the noise-caustics seamless map is shown in FIG. 1. As can be seen, the map has a caustics effect, and the textures in the map have randomness. In addition, the map is repeated and spliced in each direction, and there is no obvious joint or discontinuity between the textures. For example, see FIG. 1(a) and FIG. 1(b). Figure 9 An example of the noise-caustics seamless map is shown in FIG. 1. As can be seen, the map has a caustics effect, and the textures in the map have randomness. In addition, the map is repeated and spliced in each direction, and there is no obvious joint or discontinuity between the textures. For example, see FIG. 1(a) and FIG. 1(b). Figure 10 An example of the noise-caustics seamless map is shown in FIG. 1. As can be seen, the map has a caustics effect, and the textures in the map have randomness. In addition, the map is repeated and spliced in each direction, and there is no obvious joint or discontinuity between the textures. For example, see FIG. 1(a) and FIG. 1(b). Figure 9 An example of the noise-caustics seamless map is shown in FIG. 1. As can be seen, the map has a caustics effect, and the textures in the map have randomness. In addition, the map is repeated and spliced in each direction, and there is no obvious joint or discontinuity between the textures. For example, see FIG. 1(a) and FIG. 1(b).

[0135] Optionally, the electronic device can pre-generate a noise-caustics seamless map corresponding to each caustic dynamic effect, establish a corresponding relationship between each noise-caustics seamless map and the information of the caustic effect map, and store the corresponding relationship in a preset storage path in the electronic device or in a server. When it is determined that the rendering instruction carries the caustic identifier and the information of the caustic effect map, the corresponding relationship can be obtained from the preset storage path, and the noise-caustics seamless map corresponding to the information of the caustic effect map can be found. That is, the noise-caustics seamless map can be generated and saved offline. In this way, the noise-caustics seamless map does not need to be generated in real time, the performance overhead of the electronic device is reduced, and the operation efficiency of the electronic device is improved.

[0136] The preset flowmap is pre-generated and contains texture data of a preset motion direction and size. In other words, the vector field information contained in the preset flowmap is pre-set. Optionally, the preset flowmap can be stored in a preset storage path of the electronic device or in a server. An example of the preset flowmap is shown in FIG. 2. Figure 11 An example of the preset flowmap and the vector field information is shown in FIG. 2. In FIG. 2, Figure 11 FIG. 2(a) is a schematic diagram of the flowmap, Figure 11 FIG. 2(b) is a schematic diagram of the vector field information contained in FIG. 2(a). In FIG. 2(b), Figure 11 FIG. 2(b) is a schematic diagram of the vector field information contained in FIG. 2(a). In FIG. 2(b), Figure 11 FIG. 2(b) is a schematic diagram of the vector field information contained in FIG. 2(a). In FIG. 2(b),

[0137] Of course, the graphics rendering module obtains other rendering input data. The other rendering input data refers to other texture data in addition to the noise-caustics seamless map and the preset flowmap, as well as vertex data, material data, and the like.

[0138] S103, the graphics rendering module inputs the noise-caustics seamless map and the preset flowmap into the GPU.

[0139] Of course, the graphics rendering module also inputs the other rendering input data into the GPU.

[0140] Optionally, the graphics rendering module can generate a shader, and input each rendering input data to the GPU through the shader. The shader is a program used to control the appearance and performance of an object in a rendering process in computer graphics. The shader can include a vertex shader, a fragment shader, etc. The vertex shader is used to process vertex data. The fragment shader is used to process color data, texture data, lighting data, material data, etc.

[0141] S104, the GPU performs N1 sets of texture offset sampling on the noise caustic seamless map over time at the current sampling time, and in the sampling process, the sampling disturbance is performed through the preset flowmap to obtain N1 caustic simulation maps corresponding to the current sampling time (also referred to as the first simulation map). N1 is an integer greater than or equal to 2. Figure 1

[0142] Optionally, the GPU can pre-set the sampling time of the texture offset sampling, and at each sampling time, the noise caustic seamless map is offset over time. Optionally, the sampling time can be periodically set, that is, each sampling time can be determined according to a preset sampling period. The sampling period can be equal to the refresh period corresponding to the screen refresh rate (i.e. 1 / refresh rate). In the present embodiment, the current sampling time is taken as an example for description. The current sampling time refers to the time at which the texture offset sampling is performed on the noise caustic seamless map.

[0143] Optionally, in the process of the texture offset sampling over time, different offset amounts can be selected to obtain different caustic simulation maps to simulate the caustic effect at different times, that is, to simulate the change of the caustic over time. In the present embodiment, the noise caustic seamless map is subjected to N1 sets of texture offset sampling over time with different offset amounts to obtain N1 caustic simulation maps Figure 1 . These N1 caustic simulation maps Figure 1 can simulate the change of the caustic over time. Exemplarily, Figure 12 is a schematic diagram of a caustic simulation map provided in the present embodiment. In the diagram, Figure 12 (a), (b), (c) and (d) represent the caustic simulation maps obtained by selecting different offset amounts when the noise caustic seamless map is subjected to offset sampling over time at a certain time. Figure 1 It can be seen that, Figure 12 the caustic effects in (a), (b), (c) and (d) are not the same, for example, the density of the caustic texture is different, and the position of the caustic texture is also different. In the present embodiment, the N1 caustic simulation maps Figure 1 can simulate the change of the caustic over time similar to that shown in Figure 12 .

[0144] ​By perturbing the sampling using a pre-defined flowmap, the texture coordinates used during sampling are offset using the vector field information in the flowmap. A pre-defined flowmap can be understood as a texture map that records 2D vector field information. The texture values ​​at each point in the pre-defined flowmap, such as the color values ​​of the R and G channels, represent the direction and magnitude of the vector field at that point. Perturbing the texture sampling of the seamless caustic map using a pre-defined flowmap allows the subsequent image to exhibit quantitative flow characteristics, further increasing the randomness of caustic changes over time. This makes the simulated caustic change process more realistic and natural, resulting in better caustic dynamic effects and an improved user experience.

[0145] For example, Figure 13 This is a schematic diagram of another example of a caustic simulation texture provided in an embodiment of this application. Figure 12 The difference is, Figure 13 The caustic simulation maps in the image are the results after flowmap sampling perturbation. (Comparison) Figure 12 It can be seen that, Figure 12 The caustic texture changes are relatively regular in each image. Figure 13 In this process, after sampling perturbation, the changes in caustic texture exhibit randomness. For example, Figure 13 Figure (b) in the middle is compared to Figure 13 In Figure (a), the caustic texture becomes sparse in region 1301, while the caustic texture becomes dense in region 1302. And... Figure 13 Figure (c) in the middle is compared to Figure 13 In Figure (b), the caustic texture of 1303 in the region becomes sparse. Figure 13 The (d) diagram in the middle is compared to Figure 13 In Figure (c), the caustic texture becomes denser in region 1304. This shows that the changes in caustics after flowmap sampling perturbation are random.

[0146] In summary, step S104 can be understood as follows: the GPU offsets the texture coordinates based on the offset sampled over time and the vector field information in the preset flowmap, and then performs texture offset sampling based on the offset coordinates. For example, at the current sampling time t1, the GPU determines the offset a based on the current sampling time t1, and determines the offset b based on the vector field information in the preset flowmap. Then, it determines the offset texture coordinates based on offset a and offset b, and performs sampling based on the offset texture coordinates.

[0147] It is understandable that if this step is performed at each sampling time, then N1 caustic simulation patches can be obtained at each sampling time. Figure 1 .

[0148] S105, GPU will generate N1 caustic simulation patches corresponding to the current sampling time. Figure 1 The images are fused to obtain the caustic frame image corresponding to the current sampling time (referred to as caustic frame image 1 or the first frame image).

[0149] For N1 caustic simulation patches Figure 1 Fusion can be understood as combining N1 caustic simulation patches Figure 1 Overlay, approximately N1 caustic simulation patches Figure 1 The texture values ​​corresponding to each texture coordinate are added together.

[0150] It should be understood that for N1 caustic simulation patches Figure 1 Different overlay methods result in varying degrees of clarity, depth, and density (i.e., granularity) of the caustic textures in the resulting caustic frame images. These different overlay methods may include using different numbers of textures for overlay, and / or assigning different weights to each texture during overlay.

[0151] For example, Figure 14 This is a schematic diagram illustrating different caustic frame images obtained using different superposition methods, as provided in an embodiment of this application. It can be seen that... Figure 14 In Figure (a), the caustic texture is blurry, shallow, and sparse (the caustic grain size is coarse). Figure 14 Figure (b) in the middle is compared to Figure 14 In Figure (a), the caustic texture is clearer and deeper, but the caustic texture is sparser (the caustic grain is coarser). Figure 14 In figure (c), the caustic texture is the clearest, the caustic texture is the deepest, and the caustic texture is the densest (the caustic grain is the finest).

[0152] In this embodiment, by fusing multiple caustic simulation textures, the caustic effect simulated by the caustic frame image is more realistic and natural, improving the user's visual experience.

[0153] S106, The GPU stores the caustic frame image 1 into the frame buffer.

[0154] Of course, while rendering caustic frame image 1, the GPU also renders other view controls and stores the rendered data (called other rendering output data) into the frame buffer, which will not be described in detail here.

[0155] S107. The surfaceflinger obtains the caustic frame image 1 from the frame buffer and performs layer composition to obtain the interface to be displayed.

[0156] Specifically, the layer compositor performs layer composition on the plenoptic frame image and other rendering output data obtained from the frame buffer to obtain the to-be-displayed interface.

[0157] In S108, the layer compositor sends the to-be-displayed interface to the display.

[0158] In S109, the display displays the to-be-displayed interface.

[0159] The above describes the process of rendering the plenoptic frame image 1 by the graphics rendering module and the GPU after the lock screen application sends the rendering instruction, and the process of displaying the interface including the plenoptic frame image 1. It can be understood that at the next sampling moment, the above steps S104 to S109 are executed again, that is, a plenoptic frame image is rendered at each sampling moment, and then the interface is displayed. In the above rendering process, the plenoptic frame image at each sampling moment is composed of N1 plenoptic simulation patches, and each plenoptic simulation patch is obtained by sampling the noise plenoptic seamless patch according to the time-based texture offset. Therefore, as time goes on, the plenoptic simulation patch obtained at each sampling moment is different, and thus the plenoptic frame image obtained by fusion is different. In this way, the dynamic change effect of the plenoptic effect can be simulated. Figure 1 Figure 1 Figure 1

[0160] The above steps S104 to S109 are also referred to as a first processing process. Each sampling moment in the first processing process is also referred to as a first sampling moment, and any one of the first sampling moments is also referred to as a first target moment.

[0161] It should be noted that the electronic device needs to refresh the interface according to the preset refresh rate, and thus the lock screen application can send a refresh instruction to the graphics rendering module after the duration of the refresh period (i.e., 1 / refresh rate), and the refresh instruction is used to instruct to refresh the interface, that is, to re-render the to-be-displayed interface. In the embodiment of the present application, the lock screen application can not send a refresh instruction to the graphics rendering module in the case where it is determined that the currently displayed interface includes the plenoptic dynamic effect and no touch operation of the user acting on the current interface is detected. In this way, the electronic device continuously executes steps S104 to S109, thereby continuously rendering the plenoptic frame image and presenting the plenoptic dynamic effect.

[0162] As an optional manner, in the case where the user does not switch the interface to other interfaces, the electronic device can continuously execute the above steps S104 to S109, so that the plenoptic dynamic effect is continuously displayed, that is, the plenoptic effect is continuously in the shaking state.

[0163] ​​​As another alternative, a preset caustic life cycle can also be set, and the start time of the caustic life cycle can be the display time of the first caustic frame image 1. When the caustic life cycle ends, the steps S104-S105 are stopped, and the layer composition is performed using a fixed caustic frame image (for example, the last caustic frame image of the caustic life cycle). In this way, the effect of the caustic dynamic effect lasting for a period of time and then being static is realized. This method can save the power consumption of the electronic device.

[0164] The beneficial effects achieved by the method provided in the embodiment are summarized and described below.

[0165] 1) The method provided in the embodiment is based on noise caustic seamless mapping and preset flowmap for texture sampling over time, simulates the change process of caustic over time, forms a caustic dynamic effect, and the entire process does not require the production of caustic animation or complex mesh models, saving the performance overhead of the electronic device and improving the operating efficiency of the electronic device.

[0166] 2) In the method provided in the embodiment, the noise caustic seamless mapping can be generated offline, without the need for real-time generation, further saving the performance overhead of the electronic device and further improving the operating efficiency of the electronic device.

[0167] 3) The noise caustic seamless mapping is a picture generated by a noise algorithm and / or noise data, has randomness, is relatively realistic and natural, and moreover, the noise caustic seamless mapping is a seamless mapping, and there will be no obvious joints or discontinuous phenomena between textures. Therefore, in the method provided in the embodiment, texture offset sampling is performed based on the noise caustic seamless mapping, the obtained caustic simulation image is relatively realistic and natural, and there will be no obvious joints or discontinuous phenomena, the image effect is good, and the subsequent generation of caustic frame images is realistic and natural, the caustic dynamic effect formed is good, and the user experience is improved.

[0168] 4) The method provided in the embodiment can disturb the texture sampling of the noise caustic seamless mapping by presetting the flowmap, can make the obtained caustic simulation mapping exhibit the characteristics of quantitative flow, further increase the randomness of the caustic change over time, make the simulated caustic change process more realistic and natural, and thus make the subsequent caustic dynamic effect better and improve the user experience.

[0169] The process of texture offset sampling over time based on noise caustic seamless mapping and sampling disturbance by presetting the flowmap is further described below.

[0170] ​It can be understood that each texture coordinate in the preset flowmap corresponds to each texture coordinate in the simulated caustic map. Based on this, each texture coordinate in the preset flowmap can be respectively subjected to texture offset sampling, and the obtained texture value can represent the vector field information corresponding to the texture coordinate. Then, the texture coordinate is offset using the vector field information (i.e. the texture value sampled from the preset flowmap). Then, the noise caustic seamless map is subjected to texture offset sampling over time based on the offset texture coordinate. In this way, the disturbance in the process of offset sampling over time can be realized, and the effect of simulating flow can be generated. The following will be further described in combination with the drawings.

[0171] For example, referring to Figure 15 , the above step "S104, the GPU performs N1 sets of texture offset sampling over time on the noise caustic seamless map at the current sampling time, and in the sampling process, the sampling disturbance is performed by the preset flowmap to obtain N1 simulated caustic maps corresponding to the current sampling time Figure 1 ", includes:

[0172] S1041, the GPU performs texture offset sampling over time on each texture coordinate in the preset flowmap at the current sampling time to obtain the offset amount 1 (also referred to as the first offset amount) corresponding to each texture coordinate.

[0173] In other words, the texture offset sampling over time is performed on each texture coordinate in the preset flowmap based on the current sampling time to obtain the offset amount 1 corresponding to each texture coordinate.

[0174] Specifically, the offset amount 1 corresponding to any texture coordinate UV is represented as uv_offset, and the dynamic texture offset sampling over time of the texture coordinate UV can be represented as formula (1):

[0175] uv_offset = texture(flowmap, UV + 1.0 * t).x - 0.5 (1)

[0176] Wherein, texture() represents a texture sampling function. flowmap represents sampling a texture value from the flowmap, and in this embodiment, the flowmap is the preset flowmap.

[0177] UV represents the texture coordinates, and t represents the coordinate value corresponding to the sampling time. Optionally, the sampling time can be converted into a coordinate value through a preset conversion manner, so as to facilitate calculation with the texture coordinates. UV+1.0*t represents the texture coordinates (i.e., the sampling position) of the sampled texture value. x represents the x-axis component of the texture coordinates. It can be understood that the flowmap is a grayscale image, and the offset amounts of the R and G channels are equal, so the offset amounts obtained based on the x-axis component and the y-axis component of the flowmap are equal, and thus the x-axis component can be taken for calculation, which can simplify the calculation process. 0.5 is an adjustment parameter. As described above, the value of the texture coordinates ranges from 0 to 1, and the value of uv_offset can be adjusted to the range of [-0.5, 0.5] through the adjustment parameter 0.5. It can be understood that in other embodiments, 0.5 can also be replaced by other values.

[0178] Overall, formula (1) represents that the texture value is sampled from the texture coordinates UV+1.0*t in the flowmap, and through the adjustment of 0.5, the texture value is obtained, which is taken as the offset amount 1uv_offset corresponding to the texture coordinates UV.

[0179] It can be understood that in formula (1), the texture coordinates for obtaining the texture value are UV+1.0*t, so the texture coordinates for obtaining the texture value from the flowmap change over time, and thus the uv_offset changes. In this way, the texture offset sampling of the flowmap over time is realized.

[0180] S1042, the GPU offsets each texture coordinate according to the offset amount 1 corresponding to each texture coordinate, to obtain the offset coordinate (referred to as the disturbed coordinate 1 or the first target coordinate) corresponding to each texture coordinate.

[0181] Specifically, the disturbed coordinate 1 corresponding to any texture coordinate UV is represented as caustics_uv, and caustics_uv can be represented as the following formula (2):

[0182] caustics_uv=UV+uv_offset(2)

[0183] S1043, the GPU performs N1 sets of texture offset sampling over time on each disturbed coordinate 1 of the noise caustics seamless map according to the current sampling time, to obtain N1 caustics simulation maps corresponding to the current sampling time. Figure 1 .

[0184] As described in the above embodiment, N1 sets of texture offset sampling with different offset amounts can be performed on the noise caustics seamless map over time, to obtain N1 caustics simulation maps. Figure 1Alternatively, different offset multiples can be set to multiply the coordinates corresponding to the sampling time, so as to obtain different offsets. For example, in the N1 group of texture offset sampling, the offset multiple of the first group of texture offset sampling is 0, and the corresponding offset is t; the offset multiple of the second group of texture sampling is 1, and the corresponding offset is t; the offset multiple of the third group of texture sampling is 2, and the corresponding offset is 2t; and so on, so as to obtain N1 different offsets, and the noise caustics seamless map is sampled based on the offset. The following is described in combination with formulas.

[0185] Exemplarily, for any caustics_uv, the first group of texture offset sampling with an offset multiple of 0 is performed based on the current sampling time, and the sampled texture value is denoted as A, and the sampling process can be represented as formula (3) as follows:

[0186] A = texture(caustics, caustics_uv) (3)

[0187] Wherein, texture() represents a texture sampling function. caustics represents sampling a texture value from the noise caustics seamless map. caustics_uv represents that the texture coordinate (i.e. position) of sampling the texture value is caustics_uv, and caustics_uv is the calculation result of formula (2), that is, the disturbed coordinate 1. Overall, formula (3) represents that the texture value at the texture coordinate caustics_uv in the noise caustics seamless map is sampled.

[0188] caustics_uv can be understood as caustics_uv + 0*t, and t represents the texture coordinate corresponding to the sampling time. That is, in this group of sampling, based on the current sampling time, the texture offset sampling with an offset multiple of 0 and an offset of 0 (i.e. without offset) is performed.

[0189] It can be understood that the disturbed coordinate 1 corresponding to each texture coordinate in the noise caustics seamless map is sampled according to formula (3), so as to obtain a caustics simulation map disturbed by flowmap sampling, denoted as caustics simulation map a.

[0190] Exemplarily, for any caustics_uv, the second group of texture offset sampling with an offset multiple of 1 is performed based on the current sampling time, and the sampled texture value is denoted as B, and the sampling process can be represented as formula (4) as follows:

[0191] B = texture(caustics, caustics_uv + t) (4)

[0192] where caustics uv+t represents the texture coordinate at which the texture value is sampled, caustics uv+t = caustics uv + 1.0 * t, and caustics uv is the result of formula (2), i.e., the disturbed coordinate 1. Overall, formula (4) represents sampling the texture value from the noise caustics seamless map at the texture coordinate caustics uv+t.

[0193] where caustics uv+t represents the texture coordinate at which the texture value is sampled, caustics uv+t = caustics uv + 1.0 * t, and caustics uv is the result of formula (2), i.e., the disturbed coordinate 1. Overall, formula (4) represents sampling the texture value from the noise caustics seamless map at the texture coordinate caustics uv+t.

[0194] It can be understood that, by sampling the disturbed coordinate 1 corresponding to each texture coordinate in the noise caustics seamless map according to formula (4), a caustics simulation map b that is disturbed by the flowmap and has a time offset of 1.0 * t can be obtained.

[0195] For example, for any caustics uv, a third group of texture offset sampling with a shift multiple of 2 is performed based on the current sampling time, and the sampled texture value is represented as C. The sampling process can be represented as formula (5) as follows:

[0196] C = texture (caustics, caustics uv + 2.0 * t) (5)

[0197] where caustics uv+t represents the texture coordinate at which the texture value is sampled, caustics uv+t = caustics uv + 1.0 * t, and caustics uv is the result of formula (2), i.e., the disturbed coordinate 1. Overall, formula (4) represents sampling the texture value from the noise caustics seamless map at the texture coordinate caustics uv+t.

[0198] It can be understood that, by sampling the disturbed coordinate 1 corresponding to each texture coordinate in the noise caustics seamless map according to formula (5), a caustics simulation map c that is disturbed by the flowmap and has a time offset of 2.0 * t can be obtained.

[0199] The other group texture offset sampling based on the current sampling time is similar and will not be repeated. In this way, a plurality of caustic simulation maps can be obtained. The number of caustic simulation maps can be selected according to requirements, that is, the number of N1 can be selected according to requirements. For example, in the current scenario, the number of caustic simulation maps can be selected as 2 (that is, N1 = 2), that is, caustic simulation map a and caustic simulation map b can be generated according to the above formulas (3) and (4), respectively.

[0200] Please continue to see Figure 15 On the basis of obtaining N1 caustic simulation maps according to the above process, the above step "S105, the GPU fuses N1 caustic simulation maps corresponding to the current sampling time to obtain a caustic frame image 1", includes:

[0201] S1051, the GPU fuses N1 caustic simulation maps corresponding to the current sampling time to obtain a caustic frame image 1. Figure 1 The texture values of each texture coordinate are weighted and summed to obtain a caustic frame image 1.

[0202] Specifically, the texture value of any texture coordinate UV in the caustic frame image 1 is denoted as final. Taking N1 = 2 as an example, final can be calculated by the following formula (6):

[0203] final = A + weight * B (6)

[0204] Where weight represents the weight corresponding to B. The specific value of weight can be set according to requirements.

[0205] It can be understood that the texture value of each perturbed coordinate 1 in the noise caustic seamless map is calculated according to formula (6), and the caustic frame image 1 is obtained.

[0206] In this embodiment, the preset flowmap is sampled by texture offset sampling over time to obtain different offset amounts 1 at different times, that is, to obtain different vector field information. In this way, different vector field information is used for sampling perturbation at different times, thereby further increasing the randomness of the caustic changing over time, making the simulated caustic change process more realistic and natural, and further making the subsequent caustic dynamic effect better and improving user experience.

[0207] Process two: performing a preset operation on a lock screen interface containing a caustic dynamic wallpaper.

[0208] As described above, the caustics can change with the fluctuation of the medium or the light, for example, the caustics of the water surface can sway with the fluctuation of the water surface. In the embodiments of the present application, in the case that the lock screen wallpaper is a caustic dynamic wallpaper, the user can perform a preset operation on the lock screen interface, and the electronic device simulates the motion of the caustics according to the operation of the user in response to the preset operation, so as to improve the interest of the interaction between the electronic device and the user and improve the user experience. The following will be described in conjunction with the accompanying drawings. In addition, in the embodiments, different caustic effects can be presented in different regions of the interface according to the touch trajectory of the user, so as to further improve the authenticity of the scene simulation and improve the user experience.

[0209] Exemplarily, Figure 16 Another example interface change schematic diagram provided by the embodiments of the present application is shown in FIG. 1(a). Figure 16 As shown in FIG. 1(a), the mobile phone is in a lock screen state, and the lock screen interface 601 is displayed on the screen. The lock screen wallpaper in the lock screen interface 601 is a water surface caustic dynamic effect, and the caustics are in a static state at this time. In response to the user performing a preset operation on the lock screen interface 601, the caustic effect in the interface changes, for example, the density of the caustics, the depth of the caustics, etc. change, and the closer the position to the user's touch position, the more obvious the caustic change, and the farther the position to the user's touch position, the less obvious the caustic change, as shown in the interface 1601 in FIG. 1(b). Figure 16 Optionally, the preset operation can be a long press operation, a sliding operation or a click operation, etc., which is not limited in the present application. Figure 16 The sliding operation is taken as an example for description in FIG. 1.

[0210] The image processing method provided by the embodiments of the present application will be described below in conjunction with the above interface change process.

[0211] Figure 17 Another example flowchart of the image processing method provided by the embodiments of the present application is shown in FIG. 2, and the method comprises the following steps.

[0212] S201, the lock screen application sends a refresh instruction to the graphics rendering module in response to the user performing a preset operation on the lock screen interface, and the refresh instruction carries a touch trajectory corresponding to the preset operation.

[0213] The refresh instruction is used to instruct to refresh the interface according to the touch trajectory, that is, to re-render the interface.

[0214] S202, the graphics rendering module generates a flowmap1 according to the touch trajectory in response to the refresh instruction.

[0215] Specifically, the graphics rendering module can record the vector field information corresponding to the touch trajectory to the corresponding texture coordinates in the flowmap, so as to generate the flowmap1. In this way, the flowmap1 contains the vector field information when the user performs the touch operation.

[0216] S203, the graphic rendering module inputs the flowmap1 into the GPU.

[0217] S204, the GPU performs N2 sets of texture offset sampling over time on the noise caustic seamless map at the current sampling moment, and in the sampling process, the sampling disturbance is performed through the flowmap1 to obtain N2 caustic simulation maps corresponding to the current sampling moment (also referred to as second simulation maps). N2 is an integer greater than or equal to 2. Figure 2

[0218] S205, the GPU fuses the N2 caustic simulation maps corresponding to the current sampling moment to obtain a caustic frame image 2 (also referred to as a second frame image). Figure 2

[0219] S206, the GPU renders and stores the caustic frame image 2 into a frame buffer.

[0220] S207, a surface flinger obtains the caustic frame image 2 from the frame buffer to perform surface flinging to obtain a refreshed interface.

[0221] S208, the surface flinger sends the refreshed interface to a display.

[0222] S209, the display displays the refreshed interface.

[0223] Steps S203 to S209 are similar to steps S103 to S109, and the difference is that the sampling disturbance is performed through the flowmap1 instead of a preset flowmap.

[0224] In this embodiment, the flowmap1 is generated according to the touch track corresponding to the preset operation when the user performs the preset operation on the lock screen interface. Thus, the flowmap1 contains the vector field information related to the touch track of the user, so that the caustic dynamic effect generated based on the flowmap1 is related to the touch track of the user, and the effect that the caustic moves along the touch track of the user is presented, which improves the realism of the caustic dynamic effect and the interestingness of the interaction.

[0225] Optionally, in the above process two, a caustic life cycle can also be preset, and the start moment of the caustic life cycle can be the display moment of the first caustic frame image 2. When the caustic life cycle ends, the above steps S204 to S205 are stopped, and a fixed caustic frame image (for example, the last caustic frame image in the caustic life cycle) is used for surface flinging. In this way, the effect that the caustic dynamic effect is static after lasting for a period of time is realized. This way can save the power consumption of the electronic device. ​​

[0226] The steps S204 to S209 are also referred to as a second processing process, and each sampling time in the second processing process is also referred to as a second sampling time, and any one of the second sampling times is also referred to as a second target time.

[0227] The specific implementation process of step S204 can be similar to the processes of steps S1041 to S1043, and the difference is that flowmap1 is used in the embodiment instead of a preset flowmap. An exemplary flowchart of the image processing method provided by the embodiment of the application is shown in FIG. 4. Figure 18 An exemplary flowchart of the image processing method provided by the embodiment of the application is shown in FIG. 4. Figure 18 As shown in FIG. 4, the step "S204, GPU performs N2 sets of texture offset sampling of the noise caustic seamless map over time at the current sampling time, and in the sampling process, sampling disturbance is performed by using flowmap1 to obtain N2 caustic simulation maps corresponding to the current sampling time. Figure 2 ", includes:

[0228] S2041, GPU performs texture offset sampling of each texture coordinate in flowmap1 over time at the current sampling time to obtain an offset amount 2 (also referred to as a second offset amount) corresponding to each texture coordinate.

[0229] It can be understood that the calculation methods of the offset amount 2 and the offset amount 1 can be the same, and the difference is that the offset amount 1 is obtained by texture offset sampling based on a preset flowmap, and the offset amount 2 is obtained by texture offset sampling based on flowmap1, which are distinguished by suffixes 1 and 2. The names distinguished by suffixes in the application are similar, and details are not described herein.

[0230] S2042, GPU offsets each texture coordinate according to the offset amount 2 corresponding to the texture coordinate to obtain a disturbed coordinate 2 (also referred to as a second target coordinate) corresponding to each texture coordinate.

[0231] S2043, GPU performs N2 sets of texture offset sampling of each disturbed coordinate 2 of the noise caustic seamless map over time at the current sampling time to obtain N2 caustic simulation maps corresponding to the current sampling time. Figure 2 .

[0232] In the embodiment, the number of caustic simulation maps is N2. Optionally, N2 is an integer greater than or equal to 3. Figure 2

[0233] ​As an optional implementation, in this embodiment, when fusing caustic simulation maps, the number of caustic simulation maps used can be determined based on the gradient of the change in offset 2 corresponding to each texture coordinate. Different numbers of caustic simulation maps are selected for fusing regions with different gradient changes to achieve caustic effects with different densities in different regions, thereby further improving the realism of the caustic effect. The gradient of the change in offset 2 is also called the first gradient change.

[0234] Specifically, after step S2041 above, the method further includes:

[0235] S2044, GPU calculates the gradient of the change in offset 2 corresponding to each texture coordinate within a preset time period.

[0236] The preset time period can be a time period of a preset duration prior to the current sampling time. In a specific embodiment, the preset time period can be the time period from the previous sampling time (denoted as t0) to the current sampling time (denoted as t1). Then, for any texture coordinate UV, the gradient of the corresponding offset 2 can be calculated by formula (7):

[0237]

[0238] in, This represents the gradient of the offset 2 of texture coordinates UV during the time interval from the previous sampling time t0 to the current sampling time t1. uv_offset(t1) represents the offset 2 of texture coordinates UV at the current sampling time t1. uv_offset(t0) represents the offset 2 of texture coordinates UV at the previous sampling time t0. t1-t0 represents the time difference between the previous sampling time t0 and the current sampling time t1.

[0239] Understandable, Figure 18 In this process, step S2044 is executed between steps S2043 and S2051. In practical applications, step S2044 can also be executed at other times, such as between steps S2041 and S2042, or between steps S2042 and S2043. This application does not limit this.

[0240] Based on this, in step S205, the GPU will generate N2 caustic simulation patches corresponding to the current sampling time. Figure 2 The process of fusing the images to obtain caustic frame image 2 may include:

[0241] S2051. For any texture coordinate UV, the GPU determines whether the gradient of the offset 2 corresponding to the texture coordinate UV is greater than the preset gradient threshold; if yes, then proceed to step S2052; if no, then proceed to step S2053.

[0242] S2052, the GPU adds up the texture values corresponding to the texture coordinate UV in the N2 zoom-animated textures to obtain the texture value corresponding to the texture coordinate UV in the zoom-animated frame image 2. Figure 2

[0243] The texture value of any texture coordinate UV in the zoom-animated frame image 2 is denoted as final. Taking N2 = 3 as an example, final can be calculated by the following formula (8):

[0244] final = A + weight0 * B + weight1 * C (8)

[0245] wherein weight0 represents the weight corresponding to B, and weight1 represents the weight corresponding to C. The specific values of weight0 and weight1 can be set according to requirements. A, B, and C respectively represent the texture values corresponding to the texture coordinate UV in the zoom-animated texture. Figure 2

[0246] S2053, the GPU adds up the texture values corresponding to the texture coordinate UV in the N3 zoom-animated textures to obtain the texture value corresponding to the texture coordinate UV in the zoom-animated frame image 2. Wherein N3 is an integer greater than or equal to 2, and N3 is less than N2. Figure 2

[0247] Wherein the N3 zoom-animated textures are selected from the N2 zoom-animated textures. Figure 2 Figure 2 The selection manner is not limited. Taking N3 = 2 and N2 = 3 as an example, two zoom-animated textures with offset multiples of 0 and 1 can be selected from the three zoom-animated textures. Figure 2 Figure 2 Two zoom-animated textures with offset multiples of 1 and 2 can also be selected. Figure 2 Two zoom-animated textures with offset multiples of 0 and 2 can also be selected. Figure 2 The calculation process of this step can be referred to formula (6) above, which will not be described herein again.

[0248] That is, if the change gradient of the offset 2 corresponding to the texture coordinate UV is greater than the preset gradient threshold, a larger number of zoom-animated textures are used to superimpose the texture value of the texture coordinate. Figure 2 If the change gradient of the offset 2 corresponding to the texture coordinate UV is less than or equal to the preset gradient threshold, a smaller number of zoom-animated textures are used to superimpose the texture value of the texture coordinate. Figure 19

[0249] It can be understood that the above S2044, S2051 to S2053 are performed for each texture coordinate, and then the fusion processing of all texture coordinates can be realized to obtain the zoom-animated frame image 2. ​​​​​​

[0250] As described above, the vector field information corresponding to the user touch trajectory is contained in the flowmap1. In the process of performing the texture offset sampling of the flowmap1 over time, the change gradient of the offset sampling result of each texture coordinate (i.e., the texture value obtained by sampling, that is, uv_offset) can represent whether the touch trajectory is contained.

[0251] Specifically, if the change gradient of the offset amount 2 corresponding to the texture coordinate is large (greater than a preset gradient threshold), it indicates that the texture coordinate is likely to contain the user's touch trajectory, and therefore a large number (N2) of texture values corresponding to the texture coordinate in the caustic simulation map are superimposed to obtain the texture value corresponding to the texture coordinate. In this way, the caustic texture at the texture coordinate can be made more dense. If the change gradient of the offset amount 2 corresponding to the texture coordinate is small (less than or equal to the preset gradient threshold), it indicates that the texture coordinate is likely not to contain the user's touch trajectory, and therefore a small number (N3) of texture values corresponding to the texture coordinate in the caustic simulation map are superimposed to obtain the texture value corresponding to the texture coordinate. In this way, the caustic texture at the texture coordinate can be made more sparse. Through the above process, different regions can achieve different caustic effects with different densities according to the user's touch trajectory, which improves the interestingness and realism of the caustic effect, and thus improves the user experience.

[0252] In one embodiment, N1 is less than N2. N2 can be equal to N3, or can not be equal to N3. That is, in the scenario where the user sets the caustic dynamic wallpaper and the user does not perform the preset operation, a small number of caustic simulation maps are selected for superimposition. Since the user does not perform the preset operation, the image fusion according to the user touch trajectory is not required, and a small number of caustic simulation maps are directly selected for superimposition, which can simplify the algorithm running efficiency and save the power consumption of the electronic device.

[0253] The above embodiment takes the caustic as an example to describe the image processing method provided by the embodiments of the present application. It should be noted that the method provided by the embodiments can also be applied to other scenarios to render images with other effects to present other dynamic effects, for example, to render the dynamic effect of a wave, the flow effect of a waterfall, the dynamic effect of foam, etc. For the foam dynamic effect, the method provided by the embodiments of the present application can be used to dynamically superimpose foam maps based on the flowmap according to the change gradient of the offset amount (i.e., the change gradient of the vector) in the time domain to achieve foam frame images with different details, thereby presenting the foam dynamic effect, as shown in FIG. 8. Figure 3

[0254] ​The above describes an example of the image processing method provided by the embodiments of the present application in detail. It can be understood that, in order to implement the above functions, the electronic device comprises hardware and / or software modules corresponding to the functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered beyond the scope of the present application.

[0255] The embodiments of the present application can divide the functional modules of the electronic device according to the above method examples. For example, each functional module such as a detection unit, a processing unit, and a display unit can be divided according to each function, or two or more functions can be integrated in one module. The integrated module can be implemented in the form of hardware or a software functional module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division manner.

[0256] It should be noted that all related contents of each step involved in the above method embodiments can be cited to the functional description of the corresponding functional module, which will not be repeated here.

[0257] The electronic device provided by the embodiments of the present application is used to execute the above image processing method, and thus can achieve the same effect as the above implementation method.

[0258] In the case of using integrated units, the electronic device can further include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device. The storage module can be used to support the electronic device to execute program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0259] The processing module can be a processor or a controller. It can implement or execute various exemplary logical blocks, modules, and circuits described in combination with the disclosure of the present application. The processor can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can be a device for interacting with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0260] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device related to the embodiment can be a device with the structure as shown in the figure. ​

[0261] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the image processing method of any one of the above embodiments.

[0262] The embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer executes the related steps to realize the image processing method in the above embodiment.

[0263] In addition, the embodiment of the present application further provides a device, which can be a chip, a component or a module. The device can include a processor and a memory connected to each other. The memory is used to store computer execution instructions. When the device is running, the processor can execute the computer execution instructions stored in the memory, so that the chip executes the image processing method in the above method embodiments.

[0264] The electronic device, the computer readable storage medium, the computer program product or the chip provided in the embodiment can be used to execute the corresponding method provided above, and thus the beneficial effects thereof can refer to the beneficial effects of the corresponding method provided above, which will not be described herein again.

[0265] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0266] In the several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the modules or units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0267] ​The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0268] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0269] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical scheme of the embodiments of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0270] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method, said method being executed by an electronic device, characterized in that, The method includes: In response to the user's first operation, a first flow graph and a first texture map are obtained. The first flow graph is a texture map containing preset vector field information, and the first texture map is a seamless texture map generated based on noise. At multiple first sampling moments, the first processing procedure is executed respectively; The first processing procedure includes: At the first target time, the first texture map is sampled for texture offset over time in N1 groups, and the sampling is perturbed by the first flow graph during the sampling process to obtain N1 first simulated texture maps corresponding to the first target time, where N1 is an integer greater than 1, and the first target time is any one of the plurality of first sampling times; The N1 first simulated textures are fused to obtain the first frame image corresponding to the first target time. Display the first frame image.

2. The method according to claim 1, characterized in that, At the first target time, N1 sets of texture offset samples are performed on the first texture map over time, and the sampling is perturbed by the first flow graph during the sampling process to obtain N1 first simulated texture maps corresponding to the first target time, including: At the first target time, the texture offset of each texture coordinate in the first flow graph is sampled over time to obtain the first offset corresponding to each texture coordinate; Based on the first offset corresponding to each texture coordinate, each texture coordinate is offset to obtain the first target coordinate corresponding to each texture coordinate; Based on the first target time, N1 sets of texture offset sampling with time are performed on each first target coordinate in the first texture map to obtain the N1 first simulated texture maps.

3. The method according to claim 2, characterized in that, The step of fusing the N1 first simulated textures to obtain the first frame image corresponding to the first target time includes: The texture values ​​of each target coordinate in the N1 first simulated textures are weighted and summed to obtain the first frame image.

4. The method according to any one of claims 1 to 3, characterized in that, The first processing procedure stops execution after a preset duration.

5. The method according to any one of claims 1 to 4, characterized in that, The time interval between two adjacent first sampling moments is equal to the refresh cycle corresponding to the refresh rate of the electronic device.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Receives a user's touch operation on a first interface, the first interface including the first frame image; In response to the touch operation, the touch trajectory corresponding to the touch operation is obtained; A second flow map is generated based on the touch trajectory, and the second flow map is a texture map containing vector field information corresponding to the touch trajectory; At multiple second sampling moments, the second processing procedure is executed respectively; The second processing procedure includes: At the second target time, the first texture map is sampled for texture offset over time in N2 groups, and the sampling is perturbed by the second flow graph during the sampling process to obtain N2 second simulated texture maps corresponding to the second target time, where N2 is an integer greater than 1, and the second target time is any one of the plurality of second sampling times; The N2 second simulated textures are fused to obtain the second frame image corresponding to the second target time. The second frame image is displayed.

7. The method according to claim 6, characterized in that, At the second target time, N2 sets of texture offset samples are performed on the first texture map over time, and the sampling is perturbed by the second flow graph during the sampling process to obtain N2 second simulated texture maps corresponding to the second target time, including: At the second target time, the texture offset of each texture coordinate in the second flow graph is sampled over time to obtain the second offset corresponding to each texture coordinate; Based on the second offset corresponding to each texture coordinate, each texture coordinate is offset to obtain the second target coordinate corresponding to each texture coordinate; Based on the second target time, N2 sets of texture offset sampling with time are performed on each second target coordinate in the first texture map to obtain the N2 second simulated texture maps.

8. The method according to claim 7, characterized in that, After sampling the texture offset over time for each texture coordinate in the second flow graph at the second target time to obtain the second offset corresponding to each texture coordinate, the method further includes: Determine the gradient of the change of the second offset corresponding to the first texture coordinate within a preset time period before the first sampling time to obtain the first gradient of change, wherein the first texture coordinate is any texture coordinate in the second flow graph.

9. The method according to claim 8, characterized in that, The preset time period is the time period between the previous sampling time and the first sampling time.

10. The method according to claim 8 or 9, characterized in that, N2 is an integer greater than 2. The step of fusing the N2 second simulated textures to obtain the second frame image corresponding to the second target time includes: If the first gradient change is greater than a preset gradient threshold, then the texture values ​​corresponding to the first texture coordinates in the N2 second simulated texture images are weighted and summed to obtain the texture values ​​corresponding to the first texture coordinates in the second frame image. If the first gradient change is less than or equal to the preset gradient threshold, then the texture values ​​corresponding to the first texture coordinates in the N3 second simulated texture images are weighted and summed to obtain the texture value corresponding to the first texture coordinates in the second frame image, where N3 is an integer greater than 1 and less than N2.

11. The method according to claim 10, characterized in that, N1 is less than N2.

12. The method according to any one of claims 6 to 11, characterized in that, The first interface is either the lock screen or the desktop.

13. The method according to any one of claims 1 to 12, characterized in that, The process of obtaining the first flow map and the first texture map includes: The first flow graph and the first texture map are obtained from the memory of the server or the electronic device.

14. The method according to any one of claims 1 to 13, characterized in that, The first operation is to set the first wallpaper as the lock screen wallpaper and / or desktop wallpaper, where the first wallpaper is the dynamic wallpaper corresponding to the first texture map.

15. The method according to any one of claims 1 to 14, characterized in that, The first texture map has a caustic effect.

16. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 15.

17. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 15.

Images