Image processing method, image processing device and image processing equipment

Abstract

The invention provides an image processing method, an image processing device and image processing equipment. The image processing method comprises the following steps: creating a buffer area; a to-be-processed image is obtained, the to-be-processed image is stored in the buffer area in a first format matched with the buffer area, and the to-be-processed image comprises a first part image and a second part image; calling a first image application programming interface to generate a first texture according to a first part image in a first format, and storing the generated first texture in the buffer area in the first format; and calling a second image application programming interface to generate a second texture according to the second part image in the first format, and generating a target texture according to the generated second texture and the first texture in the first format.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and more specifically, to an image processing method, an image processing apparatus, and an image processing device. Background Art

[0002] Computer graphics is widely used in application scenarios such as videos, games, movies, animations, virtual reality and augmented reality to process images and present them on the display of electronic devices. With the rapid development of information technology, the requirements for visual display of images are also increasing.

[0003] Therefore, there is a need for improvements in image display. Summary of the invention

[0004] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0005] One of the objectives of the present disclosure is to provide an image processing method, an image processing apparatus and an image processing device.

[0006] According to a first aspect of the present disclosure, there is provided an image processing method, wherein the image processing method comprises:

[0007] Create a buffer;

[0008] Acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image;

[0009] Calling a first image application programming interface to generate a first texture according to a first partial image in a first format, and saving the generated first texture in the buffer in the first format; and

[0010] The second image application programming interface is called to generate a second texture according to the second partial image in the first format, and a target texture is generated according to the generated second texture and the first texture in the first format.

[0011] In some embodiments, calling a first image application programming interface to generate a first texture according to a first partial image in a first format includes:

[0012] Calling a shader component to obtain first shading information of a first portion of the image from the buffer according to first shader parameters declared in a first image application programming interface; and

[0013] A first texture is generated according to the first shading information.

[0014] In some embodiments, calling the second image application programming interface to generate a second texture according to the second partial image in the first format includes:

[0015] Calling the shader component to obtain second shading information of a second partial image from the buffer according to second shader parameters declared in a second image application programming interface; and

[0016] A second texture is generated according to the second shading information.

[0017] In some embodiments, the image processing method includes:

[0018] defining macros in the first image application programming interface and the second image application programming interface respectively; and

[0019] The shader parameters are declared in the shader component through the macro, so that when a first image application programming interface is called, the declared first shader parameters are obtained by expanding the macro, and when a second image application programming interface is called, the declared second shader parameters are obtained by expanding the macro.

[0020] In some embodiments, the first texture generated by calling the first image application programming interface has a second format adapted to the first image application programming interface, wherein saving the generated first texture in the first format to the buffer comprises:

[0021] converting a first texture in a second format to have the first format; and

[0022] A first texture in the first format is saved into the buffer.

[0023] In some embodiments, the second texture generated by calling the second image application programming interface has a third format adapted to the second image application programming interface, wherein generating a target texture according to the generated second texture and the first texture of the first format comprises:

[0024] Obtain a first texture in the first format from the buffer, and convert the first texture in the first format into the third format; and

[0025] A target texture is generated according to a first texture and a second texture having the same format.

[0026] In some embodiments, the buffer is a first buffer adapted for a first operating system or a second buffer adapted for a second operating system.

[0027] In some embodiments, the first imaging application programming interface is different from the second imaging application programming interface.

[0028] In some embodiments, the first graphics application programming interface is one of OpenGL ES and Vulkan, and the second graphics application programming interface is the other of OpenGL ES and Vulkan.

[0029] In some embodiments, the image processing method further includes:

[0030] The image to be processed is rendered based on the target texture, and the rendered image to be processed is displayed on a display window.

[0031] According to a second aspect of the present disclosure, there is provided an image processing device, wherein the image processing device comprises:

[0032] A creation module, configured to create a buffer;

[0033] an acquisition module, configured to acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image;

[0034] A first calling module is configured to call a first image application programming interface to generate a first texture according to a first partial image in a first format, and save the generated first texture in the buffer in the first format; and

[0035] The second calling module is configured to call a second image application programming interface to generate a second texture according to the second partial image in the first format, and to generate a target texture according to the generated second texture and the first texture in the first format.

[0036] According to a third aspect of the present disclosure, an image processing device is provided, wherein the image processing device includes a processor and a memory, the memory stores instructions, and when the instructions are executed by the processor, the steps of the image processing method as described above are implemented.

[0037] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the steps of the image processing method described above are implemented.

[0038] According to a fifth aspect of the present disclosure, a computer program product is provided, wherein the computer program product comprises instructions, and when the instructions are executed by a processor, the steps of the image processing method as described above are implemented.

[0039] Other features and advantages of the present disclosure will become more apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0041] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0042] Figure 1 A schematic diagram showing a flow chart of an image processing method according to an exemplary embodiment of the present disclosure is shown;

[0043] Figure 2 A schematic diagram of a process of calling a parameter interface according to an exemplary embodiment of the present disclosure is shown;

[0044] Figure 3 A schematic structural diagram of an image processing device according to an exemplary embodiment of the present disclosure is shown;

[0045] Figure 4 A schematic structural diagram of an image processing device according to an exemplary embodiment of the present disclosure is shown;

[0046] Figure 5 A schematic block diagram of a computer system is shown on which embodiments of the present disclosure may be implemented.

[0047] Note that in the embodiments described below, the same reference numerals are sometimes used in common between different drawings to represent the same parts or parts with the same functions, and their repeated descriptions are omitted. In this specification, similar reference numerals and letters are used to represent similar items, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0048] For ease of understanding, the positions, sizes, and ranges of various structures shown in the drawings and the like sometimes do not represent actual positions, sizes, and ranges, etc. Therefore, the disclosed invention is not limited to the positions, sizes, and ranges, etc. disclosed in the drawings and the like. In addition, the drawings are not necessarily drawn to scale, and some features may be exaggerated to show details of specific components. DETAILED DESCRIPTION

[0049] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0050] The following description of at least one exemplary embodiment is in fact merely illustrative and is in no way intended to limit the present disclosure and its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in the present disclosure. However, those skilled in the art will appreciate that they merely illustrate exemplary ways of the present disclosure that can be implemented, rather than exhaustive ways. In addition, the drawings need not be drawn to scale, and some features may be enlarged to illustrate the details of specific components.

[0051] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0052] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0053] The image application programming interface (API) is a very important component of computer graphics. It provides standardized functions and tools. In some examples, the content in the image can be rendered and displayed based on the corresponding image API in the business layer of the software system to improve the visual display effect of the image. For example, the three-dimensional scene in the image can be rendered through the corresponding image API. The image API can be divided into a low-level image API and a high-level image API. The low-level image API can include Direct3D (D3D), OpenGL, Vulkan, OpenGL ES, Meta, etc., and the high-level image API can include Object-Oriented Graphics Rendering Engine (OGRE), OpenSceneGraph (OSG), etc. The low-level image API is usually an interface for the application to interact directly with the operating system. It is close to the driver layer of the operating system and is not convenient for direct use in the business layer. Based on the corresponding low-level image API, a high-level image API can be encapsulated to form a high-level image API, so that when calling the high-level image API to render the image, the low-level image API encapsulated in the high-level image API can be called for rendering, so that specific business development can be carried out based on the high-level image API in the business layer.

[0054] However, since different image APIs usually process image data in different data formats, the format of the rendering result generated under one image API is not compatible with the format of the rendering result generated under another image API, resulting in that only a single image API can be used to render the content in the image in the business layer, and usually only one underlying image API can be used for rendering when calling a high-level image API, that is, only the rendering result of one image API can be displayed in the same display window. For example, in OGRE, usually only OpenGL or Vulkan is supported, and two underlying image APIs cannot be used at the same time.

[0055] In order to solve the above problems, the present disclosure proposes an image processing method, which creates a buffer and saves data in the buffer in a format suitable for the buffer. Different image APIs can obtain corresponding data from the buffer, so that rendering results can be shared between various image APIs, so that more than one image API can be called to render the image, thereby improving the flexibility of image rendering and enhancing the visual display effect of the image.

[0056] In an exemplary embodiment of the present disclosure, Figure 1 As shown, the image processing method may include:

[0057] Step S110, creating a buffer zone.

[0058] The buffer may be used to store data in a format adapted to the buffer (referred to herein as a first format). In some embodiments, a corresponding buffer may be created on a hardware layer, for example, the buffer may be created on a memory, or the buffer may be part of a memory space, wherein the memory may be a memory on a central processing unit (CPU).

[0059] In some embodiments, a corresponding buffer can be created according to the type of operating system. The operating system can be, for example, an Android operating system or an IOS (iPhone Operating System) operating system. Among them, different functions can be used to create a buffer adapted to the corresponding operating system. For example, the buffer can be a first buffer adapted to the first operating system, or the buffer can be a second buffer adapted to the second operating system. In the case where the first operating system is an Android operating system, the first buffer can be created by calling a corresponding function, such as the function AHardwareBuffer_allocate. In the case where the second operating system is an IOS operating system, the second buffer can be created by calling a corresponding function, such as the function CVPixelBufferCreate.

[0060] like Figure 1 As shown, in an exemplary embodiment of the present disclosure, the image processing method may further include:

[0061] Step S120, obtaining the image to be processed, and saving the image to be processed into the buffer in a first format adapted to the buffer.

[0062] The image to be processed may include a first partial image and a second partial image. The image may be a static image or a dynamic image, such as a video. When the image to be processed is a dynamic image, the first partial image and the second partial image may be images of different parts of a certain frame of the dynamic image. In a specific example, one of the first partial image and the second partial image may be an image corresponding to a canvas background (e.g., a three-dimensional scene) in the image to be processed, and the other of the first partial image and the second partial image may be an image corresponding to screen content (e.g., content other than the three-dimensional scene) in the image to be processed.

[0063] In some embodiments, shading information related to the image to be processed may be stored in a buffer, wherein the shading information may include, for example, data corresponding to shader parameters at a specified pixel coordinate position in the image. Shader parameters may include, for example, parameters related to image-based lighting (IBL), such as IBL intensity, IBL angle, and the like.

[0064] like Figure 1 As shown, in an exemplary embodiment of the present disclosure, the image processing method may further include:

[0065] Step S130, calling a first image application programming interface to generate a first texture according to a first partial image in a first format, and saving the generated first texture in a buffer in the first format; and

[0066] Step S140 , calling a second image application programming interface to generate a second texture according to the second partial image in the first format, and generating a target texture according to the generated second texture and the first texture in the first format.

[0067] The first image API may be a low-level image API, such as any one of Direct3D (D3D), OpenGL, Vulkan, OpenGL ES, and Meta. Alternatively, the first image API may be a high-level image API, such as any one of Object-Oriented Graphics Rendering Engine (OGRE) and OpenSceneGraph (OSG). The second image API may be a low-level image API, such as any one of Direct3D (D3D), OpenGL, Vulkan, OpenGLES, and Meta. Alternatively, the second image API may be a high-level image API, such as any one of Object-Oriented Graphics Rendering Engine (OGRE) and OpenSceneGraph (OSG).

[0068] In some embodiments, when the first image API is called, the first partial image in the first format can be obtained from the buffer, so as to convert the first partial image in the first format into a second format compatible with the first image API according to a processing component preset in the first image API. In this way, the first partial image in the first format saved in the buffer can be converted into a second format that can be processed by the first image API, so that when the first image API is called, the corresponding first texture can be generated according to the first partial image in the second format.

[0069] Similarly, when the second image API is called, the second partial image in the first format can be obtained from the buffer, so as to convert the second partial image in the first format into a third format compatible with the second image API according to the processing component preset in the second image API. In this way, the second partial image in the first format stored in the buffer can be converted into a third format that can be processed by the second image API, so that when the second image API is called, the corresponding second texture can be generated according to the second partial image in the third format.

[0070] In some embodiments, the first image API is different from the second image API. In this way, two different image APIs can be used to render the image and the rendering results of the two different image APIs can be displayed on the display window at the same time, so that the advantages of each image API can be fully utilized. For example, one image API has a good rendering effect on a three-dimensional scene, and the other image API has a good rendering effect on a non-three-dimensional scene. Then, the rendering capabilities of the two image APIs can be fully utilized, and one image API is used to render the three-dimensional scene in the image, and the other image API is used to render the non-three-dimensional scene in the image, thereby improving the flexibility of rendering and enhancing the visual display effect of the image. In a specific example, the first image API can be one of OpenGL ES and Vulkan, and the second image API can be the other of OpenGL ES and Vulkan.

[0071] Different image APIs call different functions when calling the image API to get data from the buffer and generate textures. In addition, different operating systems call different functions when calling the image API to get data from the buffer and generate textures.

[0072] In some embodiments, in the case of calling a first image API, a first texture may be generated by calling a shader component. Specifically, calling a first image application programming interface to generate a first texture according to a first partial image in a first format may include: calling a shader component to obtain first shading information of the first partial image from a buffer according to a first shader parameter declared in the first image API, and generating a first texture according to the first shading information. Similarly, in the case of calling a second image API, a second texture may also be generated by calling a shader component. Specifically, calling a second image application programming interface to generate a second texture according to a second partial image in a first format may include: calling a shader component to obtain second shading information of the second partial image from a buffer according to a second shader parameter declared in the second image API, and generating a second texture according to the second shading information. In a specific example, in the case of calling a shader component, a texture may be generated by processing the shading information of the image based on a processing component pre-set in the shader component.

[0073] In different image APIs, the versions of shader components may be different. Accordingly, the ways of declaring shader parameters in different image APIs may be different, so that when calling the shader component, the ways of obtaining data from the buffer and writing data in the buffer according to the same shader parameters in different image APIs may also be different. In order to ensure that the shading information corresponding to the first part of the image and the shading information corresponding to the second part of the image can be effectively obtained from the buffer according to the same shader parameters when calling the first image API and the second image API, respectively, to avoid the shading information obtained from the buffer according to the shader parameters when calling the first image API not corresponding to the first part of the image, or the shading information obtained from the buffer according to the shader parameters when calling the second image API not corresponding to the second part of the image, it is necessary to adapt the first shader parameters declared in the first image API and the second shader parameters declared in the second image API to ensure that the first texture generated by calling the first image API and the second texture generated by calling the second image API can be effectively adapted and generate the target texture later.

[0074] In some embodiments, parameters declared in different graphics APIs can be adapted through macros, where the macro is a collection of pre-written commands. Specifically, macros can be defined in a first graphics API and a second graphics API, respectively, and shader parameters can be declared through the macros in a shader component, so that the declared first shader parameters can be obtained by expanding the macros when the first graphics API is called, and the declared second shader parameters can be obtained by expanding the macros when the second graphics API is called.

[0075] In a specific example, an example of defining a macro in OpenGLES may be as follows:

[0076] #define BEGIN_STRUCT(setNo, bindingNo, structName);

[0077] #define STRUCT_MEMBER(type, name) uniform type name;

[0078] #define END_STRUCT(name);

[0079] An example of defining a macro in Vulkan could look like this:

[0080] #define BEGIN_STRUCT(setNo, bindingNo, structName) layout(set =setNo, binding = bindingNo, std140) uniform structName {

[0081] #define STRUCT_MEMBER(type, name) type name;

[0082] #define END_STRUCT(name)} name.

[0083] An example of declaring shader parameters via macros in a shader component could look like this:

[0084] BEGIN_STRUCT(0, 4, _IBL);

[0085] STRUCT_MEMBER(float, IBLIntensity);

[0086] STRUCT_MEMBER(float, IBLAngle);

[0087] END_STRUCT(ibl).

[0088] Thus, when calling OpenGLES, by expanding the macro, we can get:

[0089] uniform float IBLIntensity;

[0090] uniform float IBLAngle;

[0091] In the case of calling Vulkan, by expanding the macro we get:

[0092] layout(set = 0, binding = 4, std140) uniform _IBL {

[0093] float IBLIntensity;

[0094] float IBLAngle;

[0095] In this way, the shader parameters IBLIntensity and IBLAngle declared in OpenGLES and Vulkan can be effectively guaranteed to be compatible.

[0096] The first texture generated by calling the first image API has a second format adapted to the first image API, and the second texture generated by calling the second image API has a third format adapted to the second image API. In some embodiments, the first texture in the second format can be converted to a first format adapted to the buffer, and the first texture in the first format is saved in the buffer. In this way, the second image API can read the first texture in the first format from the buffer, so as to generate a target texture in combination with the second texture later.

[0097] In some embodiments, generating a target texture based on the generated second texture and the first texture in the first format may include acquiring the first texture in the first format from the buffer, converting the first texture in the first format into a third format compatible with the second image API, and generating the target texture based on the first texture and the second texture in the same format. In this way, the first texture and the second texture in the same format may be integrated in the second image API to generate the target texture, thereby integrating rendering results of different image APIs.

[0098] In some embodiments, the second texture generated by calling the second image API can also be saved in the buffer in the first format, and in the case of calling the first image API, the target texture is generated according to the first texture generated by calling the first image API and the second texture of the first format. Specifically, the second texture with the third format generated by calling the second image API can be converted to a first format that is compatible with the buffer, and the second texture of the first format is saved in the buffer. Then, in the case of calling the first image API, it can include obtaining the second texture of the first format from the buffer, converting the second texture of the first format to a second format that is compatible with the first image API, and generating the target texture according to the first texture and the second texture of the same format. In this way, the first image API can read the second texture of the first format from the buffer, and can integrate the first texture and the second texture of the same format in the first image API to generate the target texture. In other words, the integration of the first texture and the second texture to generate a target texture can be completed not only in the second image API, but also in the first image API, which may depend on the target format of the target texture to be generated. For example, when the target texture is expected to have a second format compatible with the first image API, the first texture and the second texture can be integrated under the first image API; when the target texture is expected to have a third format compatible with the second image API, the first texture and the second texture can be integrated under the second image API.

[0099] In some embodiments, the image to be processed can be rendered based on the target texture, and the rendered image to be processed can be displayed on the display window. In this way, it is possible to display the rendering results of more than one image API on the same display window. In a specific example, the rendered image to be processed can be displayed on the display of an electronic device. The electronic device can be, for example, a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., without limitation.

[0100] In some embodiments, the first image API and the second image API may be encapsulated to generate a unified image API, so that when the unified image API is called to render the image to be processed, the first image API and the second image API may be called to generate the first texture and the second texture to generate the rendering result of the image to be processed. In this way, the business layer can be facilitated to carry out specific business development by calling the unified image API, simplifying the workload of the business layer, so that business developers no longer need to care about the implementation details of the underlying image API.

[0101] In some embodiments, a parameter interface may be designed for a unified graphics API so that the first graphics API and the second graphics API can write corresponding data at a location corresponding to a shader parameter via the parameter interface, so that when a shader component is called, the shading information obtained from the buffer can be processed according to the data at the corresponding location of the shader parameter to generate a corresponding texture. The location corresponding to the shader parameter may be the location of the shader parameter in the memory of a graphics processing unit (GPU).

[0102] In a specific example, the following parameter interface can be designed by the unified image API:

[0103] SetShaderParam(const std::string& name, void* constant, uint32_tsize).

[0104] like Figure 2As shown, when calling SetShaderParam at the business layer, the parameter name ParameterName and the data to be written Data can be input. Then, in OpenGL ES, the location of the parameter corresponding to ParameterName in the GPU memory can be obtained through the functions glGetActiveAttrib and glGetAttribLocation, and then the data Data can be written at the corresponding location according to the function glUniformXXX. In Vulkuan, the location of the parameter corresponding to ParameterName in the GPU memory can be obtained through the spirv_reflection library. Specifically, the structure SpvReflectDescriptorBinding can be obtained by calling the function spv_reflect::ShaderModule::EnumerateDescriptorBindings. The location and data size of the parameter corresponding to ParameterName in the parameter buffer of the GPU memory can be obtained by parsing the structure SpvReflectDescriptorBinding, and recorded in the structures ShaderMemberDesc and ShaderStructDesc. Then, the corresponding Data is copied to the corresponding location in the parameter buffer according to the data recorded in the structure ShaderMemberDesc by calling the memcpy function. Among them, the structures ShaderMemberDesc and ShaderStructDesc can be as follows:

[0105] struct ShaderMemberDesc

[0106] {

[0107] std::string name;

[0108] uint32_t offset;

[0109] uint32_t size;

[0110] };

[0111] struct ShaderStructDesc

[0112] {

[0113] ShaderMemberDescstructDesc;

[0114] std::vector <shadermemberdesc>members;

[0115] }.

[0116] In this way, when the business layer calls SetShaderParam, it can write data at the position corresponding to the parameter through the corresponding parameter name.

[0117] According to another aspect of the present disclosure, an image processing device is further provided, wherein the image processing device may include a module configured to execute the image processing method described in any of the aforementioned embodiments of the present disclosure.

[0118] like Figure 3 As shown, an image processing device 200 according to an exemplary embodiment of the present disclosure may include a creation module 210, an acquisition module 220, a first calling module 230, and a second calling module 240. The creation module 210 may be configured to create a buffer. The acquisition module 220 may be configured to acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image. The first calling module 230 may be configured to call a first image application programming interface to generate a first texture according to a first partial image in a first format, and save the generated first texture into the buffer in a first format. The second calling module 240 may be configured to call a second image application programming interface to generate a second texture according to a second partial image in a first format, and generate a target texture according to the generated second texture and the first texture in the first format.

[0119] According to another aspect of the present disclosure, an image processing device is also provided. Figure 4 As shown, in an exemplary embodiment of the present disclosure, the image processing device 300 may include a processor 310 and a memory 320. The memory 320 stores instructions, which, when executed by the processor 310, implement the steps of the image processing method described in any of the foregoing embodiments of the present disclosure.

[0120] The processor 310 may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, to implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc., and may be an X86 architecture or an ARM architecture, etc.

[0121] The memory 320 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronously connected dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0122] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the steps of the image processing method described in any of the foregoing embodiments of the present disclosure can be implemented.

[0123] Similarly, the non-transitory computer-readable storage medium in the embodiments of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the non-transitory computer-readable storage medium described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0124] According to another aspect of the present disclosure, a computer program product is further provided. The computer program product includes instructions. When the instructions are executed by a processor, the steps of the image processing method described in any of the aforementioned embodiments of the present disclosure can be implemented.

[0125] Instructions may be any set of instructions to be executed directly by one or more processors, such as machine code, or any set of instructions to be executed indirectly, such as a script. The terms "instructions", "applications", "processes", "steps", and "programs" (including "computer programs") herein may be used interchangeably herein. Instructions may be stored in an object code format for direct processing by one or more processors, or in any other computer language, including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. The functions, methods, and routines of instructions are explained in more detail elsewhere herein.

[0126] Figure 5 A schematic block diagram of a computer system 400 on which an embodiment of the present disclosure may be implemented is shown. The computer system 400 includes a bus 410 or other communication mechanism for transmitting information, and a processing device 420 coupled to the bus 410 for processing information. The computer system 400 also includes a memory 430 coupled to the bus 410 for storing instructions to be executed by the processing device 420, which may be a random access memory (RAM) or other dynamic storage device. The memory 430 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processing device 420. The computer system 400 may also include a read-only memory (ROM) 440 or other static storage device coupled to the bus 410 for storing static information and instructions for the processing device 420. A storage device 450 such as a magnetic disk or optical disk is provided and coupled to the bus 410 for storing information and instructions. The computer system 400 may be coupled to an output device 460 for providing output to a user via the bus 410, such as, but not limited to, a display (such as a cathode ray tube (CRT) or a liquid crystal display (LCD)), a speaker, etc. An input device 470 such as a keyboard, a mouse, a microphone, etc. is coupled to the bus 410 for transmitting information and command selections to the processing device 420. The computer system 400 may perform embodiments of the present disclosure. In accordance with certain implementations of the present disclosure, the results are provided by the computer system 400 in response to the processing device 420 executing one or more sequences of one or more instructions contained in the memory 430. Such instructions may be read from another computer-readable medium such as the storage device 450 into the memory 430. The execution of the sequence of instructions contained in the memory 430 causes the processing device 420 to perform the methods described herein. Alternatively, hard-wired circuitry may be used instead of software instructions or in combination with software instructions to implement the present teachings. Therefore, implementations of the present disclosure are not limited to any specific combination of hardware circuitry and software. In various embodiments, computer system 400 can be connected to one or more other computer systems such as computer system 400 across a network via network interface 480 to form a networked system. The network can include a private network or a public network such as the Internet. In a networked system, one or more computer systems can store data and supply data to other computer systems. The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to processing device 420 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical disks or disks such as storage device 450. Volatile media include dynamic memories such as memory 430. Transmission media include coaxial cables, copper wires, and optical fibers, including wiring that includes bus 410.Common forms of computer-readable media or computer program products include, for example, floppy disks, flexible disks, hard disks, tapes, or any other magnetic media, CD-ROMs, digital video disks (DVDs), Blu-ray disks, any other optical media, thumb drives, memory cards, RAMs, PROMs and EPROMs, flash EPROMs, any other memory chips or boxes, or any other tangible media from which a computer can read. Various forms of computer-readable media may be involved when carrying one or more sequences of one or more instructions to processing device 420 for execution. For example, instructions may initially be carried on a disk of a remote computer. The remote computer may load instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 400 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 410 may receive the data carried in the infrared signal and place the data on bus 410. Bus 410 carries the data to memory 430, and processing device 420 retrieves instructions from memory 430 and executes the instructions. Optionally, the instructions received by memory 430 may be stored on storage device 450 either before or after execution by processing device 420 .

[0127] According to various embodiments, instructions configured to be executed by the processing device 420 to perform the method are stored on a computer-readable medium. A computer-readable medium may be a device that stores digital information. For example, a computer-readable medium includes a compact disk read-only memory (CD-ROM) as known in the art for storing software. The computer-readable medium is accessed by a processor adapted to execute the instructions configured to be executed.

[0128] In the technical solution disclosed in the present invention, textures generated under different image APIs can be converted through a buffer to achieve texture sharing, so that textures generated under various image APIs can be integrated in one image API, so that more than one image API can be called to render the image, and multiple image APIs can be used for rendering in the same display window, thereby improving the flexibility of image rendering, enhancing the visual display effect of the image, and facilitating modular expansion, upgrading or replacement of image APIs in some large projects.

[0129] As used herein, the word "exemplary" means "serving as an example, instance, or illustration" rather than as a "model" to be exactly copied. Any implementation described as exemplary herein is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, the present disclosure is not to be limited by any expressed or implied theory given in the technical field, background, summary, or detailed description.

[0130] As used herein, the term "substantially" is meant to include any minor variations due to design or manufacturing imperfections, device or component tolerances, environmental effects, and / or other factors. The term "substantially" also allows for deviations from a perfect or ideal condition due to parasitic effects, noise, and other practical considerations that may exist in actual implementations.

[0131] In addition, "first", "second" and the like may also be used herein for reference purposes only and are not intended to be limiting. For example, the terms "first", "second" and other such numerical terms referring to structures or elements do not imply a sequence or order unless the context clearly indicates otherwise.

[0132] It should also be understood that when the term "include / comprises" is used in this document, it indicates the presence of the specified features, integers, steps, operations, units and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, units and / or components and / or their combinations.

[0133] In this disclosure, the term "provide" is used in a broad sense to cover all ways of obtaining an object, and thus "providing an object" includes but is not limited to "purchasing", "preparing / manufacturing", "arranging / setting up", "installing / assembling", and / or "ordering" an object, etc.

[0134] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0135] Those skilled in the art will appreciate that the boundaries between the above operations are merely illustrative. Multiple operations can be combined into a single operation, a single operation can be distributed in additional operations, and operations can be performed at least partially overlapping in time. Moreover, alternative embodiments may include multiple instances of a specific operation, and the order of operations may be changed in various other embodiments. However, other modifications, variations, and replacements are equally possible. Aspects and elements of all embodiments disclosed above may be combined in any manner and / or in combination with aspects or elements of other embodiments to provide multiple additional embodiments. Therefore, this specification and accompanying drawings should be considered illustrative, not restrictive.

[0136] In addition, the embodiments of the present disclosure may also include the following examples:

[0137] 1. An image processing method, wherein the image processing method comprises:

[0138] Create a buffer;

[0139] Acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image;

[0140] Calling a first image application programming interface to generate a first texture according to a first partial image in a first format, and saving the generated first texture in the buffer in the first format; and

[0141] The second image application programming interface is called to generate a second texture according to the second partial image in the first format, and a target texture is generated according to the generated second texture and the first texture in the first format.

[0142] 2. The image processing method according to item 1, wherein calling a first image application programming interface to generate a first texture according to a first partial image in a first format comprises:

[0143] Calling a shader component to obtain first shading information of a first portion of the image from the buffer according to first shader parameters declared in a first image application programming interface; and

[0144] A first texture is generated according to the first shading information.

[0145] 3. The image processing method according to 2, wherein calling the second image application programming interface to generate the second texture according to the second partial image in the first format comprises:

[0146] Calling the shader component to obtain second shading information of a second partial image from the buffer according to second shader parameters declared in a second image application programming interface; and

[0147] A second texture is generated according to the second shading information.

[0148] 4. The image processing method according to 3, wherein the image processing method comprises:

[0149] defining macros in the first image application programming interface and the second image application programming interface respectively; and

[0150] The shader parameters are declared in the shader component through the macro, so that when a first image application programming interface is called, the declared first shader parameters are obtained by expanding the macro, and when a second image application programming interface is called, the declared second shader parameters are obtained by expanding the macro.

[0151] 5. The image processing method according to 1, wherein the first texture generated by calling the first image application programming interface has a second format adapted to the first image application programming interface, and wherein saving the generated first texture in the first format to the buffer comprises:

[0152] converting a first texture in a second format to have the first format; and

[0153] A first texture in the first format is saved into the buffer.

[0154] 6. The image processing method according to item 1, wherein the second texture generated by calling the second image application programming interface has a third format adapted to the second image application programming interface, and wherein generating the target texture according to the generated second texture and the first texture of the first format comprises:

[0155] Obtain a first texture in the first format from the buffer, and convert the first texture in the first format into the third format; and

[0156] A target texture is generated according to a first texture and a second texture having the same format.

[0157] 7. The image processing method according to 1, wherein the buffer is a first buffer adapted to a first operating system or a second buffer adapted to a second operating system.

[0158] 8. The image processing method according to 1, wherein the first image application programming interface is different from the second image application programming interface.

[0159] 9. The image processing method according to 8, wherein the first image application programming interface is one of OpenGL ES and Vulkan, and the second image application programming interface is the other of OpenGL ES and Vulkan.

[0160] 10. The image processing method according to item 1, wherein the image processing method further comprises:

[0161] The image to be processed is rendered based on the target texture, and the rendered image to be processed is displayed on a display window.

[0162] 11. An image processing device, wherein the image processing device comprises:

[0163] A creation module, configured to create a buffer;

[0164] an acquisition module, configured to acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image;

[0165] A first calling module is configured to call a first image application programming interface to generate a first texture according to a first partial image in a first format, and save the generated first texture in the buffer in the first format; and

[0166] The second calling module is configured to call a second image application programming interface to generate a second texture according to the second partial image in the first format, and to generate a target texture according to the generated second texture and the first texture in the first format.

[0167] 12. An image processing device, wherein the image processing device comprises a processor and a memory, the memory stores instructions, and when the instructions are executed by the processor, the steps of the image processing method according to any one of 1 to 10 are implemented.

[0168] 13. A non-transitory computer-readable storage medium, wherein instructions are stored on the non-transitory computer-readable storage medium, and when the instructions are executed by a processor, the steps of the image processing method according to any one of 1 to 10 are implemented.

[0169] 14. A computer program product, wherein the computer program product comprises instructions, and when the instructions are executed by a processor, the steps of the image processing method according to any one of 1 to 10 are implemented.

[0170] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. The various embodiments disclosed herein may be combined in any manner without departing from the spirit and scope of the present disclosure. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.< / shadermemberdesc>

Claims

1. An image processing method, characterized in that: The image processing method comprises: Create a buffer; Acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image; Calling a first image application programming interface to generate a first texture according to a first partial image in a first format, and saving the generated first texture in the buffer in the first format; and The second image application programming interface is called to generate a second texture according to the second partial image in the first format, and a target texture is generated according to the generated second texture and the first texture in the first format.

2. The image processing method according to claim 1, characterized in that: Calling a first image application programming interface to generate a first texture according to a first partial image in a first format includes: calling a shader component to obtain first shading information of the first partial image from the buffer according to a first shader parameter declared in the first image application programming interface, and generating a first texture according to the first shading information; and / or Calling the second image application programming interface to generate a second texture based on the second partial image in the first format includes: calling the shader component to obtain second shading information of the second partial image from the buffer according to second shader parameters declared in the second image application programming interface, and generating a second texture based on the second shading information.

3. The image processing method according to claim 2, characterized in that: The image processing method comprises: defining macros in the first image application programming interface and the second image application programming interface respectively; and The shader parameters are declared in the shader component through the macro, so that when a first image application programming interface is called, the declared first shader parameters are obtained by expanding the macro, and when a second image application programming interface is called, the declared second shader parameters are obtained by expanding the macro.

4. The image processing method according to claim 1, characterized in that: The first texture generated by calling the first image application programming interface has a second format adapted to the first image application programming interface, wherein saving the generated first texture in the first format to the buffer comprises: converting the first texture in the second format to have the first format, and saving the first texture in the first format to the buffer; and / or The second texture generated by calling the second image application programming interface has a third format adapted to the second image application programming interface, wherein generating a target texture based on the generated second texture and the first texture in the first format includes: obtaining the first texture in the first format from the buffer, converting the first texture in the first format to have the third format, and generating the target texture based on the first texture and the second texture with the same format.

5. The image processing method according to claim 1, characterized in that: The buffer is a first buffer adapted to the first operating system or a second buffer adapted to the second operating system; and / or The first graphics application programming interface is different from the second graphics application programming interface; and / or The first graphics application programming interface is one of OpenGL ES and Vulkan, and the second graphics application programming interface is the other of OpenGL ES and Vulkan.

6. The image processing method according to claim 1, characterized in that: The image processing method further comprises: The image to be processed is rendered based on the target texture, and the rendered image to be processed is displayed on a display window.

7. An image processing device, characterized in that: The image processing device comprises: A creation module, configured to create a buffer; an acquisition module, configured to acquire an image to be processed, and save the image to be processed into the buffer in a first format adapted to the buffer, wherein the image to be processed includes a first partial image and a second partial image; A first calling module is configured to call a first image application programming interface to generate a first texture according to a first partial image in a first format, and save the generated first texture in the buffer in the first format; and The second calling module is configured to call a second image application programming interface to generate a second texture according to the second partial image in the first format, and to generate a target texture according to the generated second texture and the first texture in the first format.

8. An image processing device, characterized in that: The image processing device comprises a processor and a memory, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores instructions, and when the instructions are executed by a processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises instructions, and when the instructions are executed by a processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.

Images