An image processing method, apparatus, electronic device, and storage medium

Abstract

The method comprises: obtaining configuration information matched with special effect editing when the special effect editing is performed on an original image, the configuration information comprising a deep learning inference node, a pre-processing function node and a post-processing function node used for performing the special effect editing on the original image; calling processing logic of the pre-processing function node according to the configuration information to obtain input data; obtaining output data through an algorithm model corresponding to the deep learning inference node; and calling processing logic of the post-processing function node according to the configuration information to obtain a target image to which the special effect is added. The method can reuse the same processing logic when different special effects are edited on the original image, thereby reducing the size of an image application program and improving development efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of information technology, and in particular, to an image processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the continuous development of terminal technology and network technology, more and more application programs adapted to terminals have emerged. For example, an image application program can be used to edit original images in multiple types of special effects to add different special effects to the original images, thereby improving the display effect of the images.

[0003] Currently, when a certain type of special effect editing is performed on an original image, a configuration file corresponding to the type of special effect editing needs to be used to implement the special effect editing. The configuration file stores processing logic required to implement the type of special effect editing. However, due to the similarity between different special effects, there can be a lot of repeated processing logic in the configuration files corresponding to different types of special effect editing. As the number of special effects increases, the existing processing method will result in more and more repeated processing logic, thereby increasing the size of the image application program package. In addition, storing complete processing logic for implementing a certain type of special effect editing in each configuration file will result in a large amount of development work, thereby reducing the development efficiency. SUMMARY

[0004] To solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide an image processing method, device, electronic device, and storage medium. When different special effects are edited on an original image, the same processing logic is reused, thereby reducing the size of the image application program and improving the development efficiency.

[0005] The embodiments of the present disclosure provide an image processing method, which comprises:

[0006] When a special effect is edited on an original image, configuration information matched with the special effect editing is obtained, the configuration information comprising a deep learning inference node used to edit the special effect on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node;

[0007] According to the configuration information, the processing logic of the pre-processing function node is called to transform the original image through the processing logic of the pre-processing function node, and input data meeting the input requirements of an algorithm model corresponding to the deep learning inference node is obtained;

[0008] The original image is edited through the algorithm model corresponding to the deep learning inference node based on the input data, and output data is obtained;

[0009] According to the configuration information, processing logic of the post-processing function node is invoked to transform the output data by the processing logic of the post-processing function node, and a target image with special effects is obtained.

[0010] The embodiments of the present disclosure further provide an image processing device, which comprises:

[0011] The acquisition module is configured to acquire configuration information matched with the special effect editing when the original image is edited with special effects, the configuration information comprising a deep learning inference node used for editing the original image with special effects, and a pre-processing function node and a post-processing function node associated with the deep learning inference node.

[0012] The first invocation module is configured to invoke processing logic of the pre-processing function node according to the configuration information, to transform the original image by the processing logic of the pre-processing function node, and obtain input data meeting input requirements of an algorithm model corresponding to the deep learning inference node.

[0013] The second invocation module is configured to edit the original image with special effects by the algorithm model corresponding to the deep learning inference node based on the input data, and obtain output data.

[0014] The third invocation module is configured to invoke processing logic of the post-processing function node according to the configuration information, to transform the output data by the processing logic of the post-processing function node, and obtain a target image with special effects.

[0015] The embodiments of the present disclosure further provide an electronic device, which comprises:

[0016] One or more processors;

[0017] A storage device configured to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described above.

[0019] The embodiments of the present disclosure further provide a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the image processing method as described above.

[0020] The technical solutions provided by the embodiments of the present disclosure have at least the following advantages compared with the prior art:

[0021] The image processing method provided by the embodiments of the present disclosure, when performing special effect editing on an original image, acquires configuration information matched with the special effect editing, the configuration information including a deep learning inference node used for performing special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node; the processing logic of the pre-processing function node is called according to the configuration information, so that the original image is transformed by the processing logic of the pre-processing function node, and input data meeting the input requirement of an algorithm model corresponding to the deep learning inference node is obtained; the original image is edited by the algorithm model corresponding to the deep learning inference node based on the input data, and output data is obtained; the processing logic of the post-processing function node is called according to the configuration information, so that the output data is transformed by the processing logic of the post-processing function node, and a target image to which a special effect is added is obtained. In other words, the configuration information matched with each special effect editing only includes a deep learning inference node used for performing this special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node, but does not include the algorithm model corresponding to the deep learning inference node and the processing logic of the pre-processing function node and the post-processing function node; the algorithm model corresponding to the deep learning inference node and the processing logic of the pre-processing function node and the post-processing function node are stored separately from the configuration information, so that when the original image is edited by different special effects, the same processing logic involved can be reused, so as to reduce the data amount of the configuration information matched with each special effect editing, and further reduce the overall data amount of the application program used for image processing, and improve the development efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent by describing in detail the following specific embodiments with reference to the attached drawings. The same or similar components are denoted by the same or similar reference numerals throughout the drawings. It is to be understood that the drawings are schematic, and the components and elements are not necessarily drawn to scale.

[0023] Figure 1 A flowchart of an image processing method in an embodiment of the present disclosure;

[0024] Figure 2 A schematic diagram of a processing link for implementing different editing special effects in an embodiment of the present disclosure;

[0025] Figure 3 A structural schematic diagram of an image processing device in an embodiment of the present disclosure;

[0026] Figure 4 A structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a terminal, including but not limited to smartphones, PDAs, tablets, wearable devices with displays, desktop computers, laptops, all-in-one computers, smart home devices, etc.

[0034] like Figure 1 As shown, the method specifically includes the following steps:

[0035] In step 110, configuration information matched with the special effect editing is obtained when the special effect editing is performed on the original image, the configuration information including a deep learning inference node used for performing the special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node.

[0036] The original image can be an image frame in a video to be edited, or a picture material imported by a user from a local album, or a picture material downloaded from the Internet, etc.

[0037] The purpose of performing the special effect editing on the original image can be to add some special effects to the original image, so that the visual effect of the original image is better and more interesting, etc. For example, the purpose of performing the special effect editing on the original image is to transform the face of a person in the original image into a cartoon face, a funny face, or to transform the mouth of the person into an exaggerated large mouth, etc.

[0038] Specifically, the user can trigger a special effect control in the interface to trigger the operation of performing the special effect editing on the original image. In other words, the special effect editing can correspond to a specific special effect control, for example, special effect control 1 corresponds to special effect editing A, special effect control 2 corresponds to special effect editing B, etc. When the user triggers a specific special effect control (for example, special effect control 1), the configuration information associated with the special effect control is obtained, that is, the configuration information matched with the special effect editing A is obtained. The configuration information can be stored in the terminal or in the server.

[0039] Optionally, the obtaining of the configuration information matched with the special effect editing includes: determining a configuration file having a preset binding relationship with the special effect editing; and reading the configuration information from the configuration file.

[0040] The configuration information includes the deep learning inference node used for performing the special effect editing on the original image, and the pre-processing function node and the post-processing function node associated with the deep learning inference node. However, the configuration information does not include the algorithm model corresponding to the deep learning inference node, and the processing logic of the pre-processing function node and the processing logic of the post-processing function node. That is, the algorithm model corresponding to the deep learning inference node, the processing logic of the pre-processing function node and the processing logic of the post-processing function node are stored separately from the configuration information. The advantage of this design is that, on the one hand, the same processing logic involved in performing different special effect editing on the original image can be reused; on the other hand, compared with the way of writing the algorithm model corresponding to the deep learning inference node, and the processing logic of the pre-processing function node and the processing logic of the post-processing function node in the configuration information, the technical solution of the embodiment of the present disclosure can greatly reduce the data amount of the configuration information, and when the types of special effect editing are more, the overall data amount of the application program can be reduced, and the development efficiency can be improved.

[0041] Taking the first effect editing, which transforms a human face in the original image into a cartoon face, and the second effect editing, which transforms a human face in the original image into an animal face, as examples, both effects require the same preprocessing operations on the original image, such as identifying the region where the human face is located in the original image. The processing logic used to identify the region where the human face is located in the original image is abstracted as preprocessing function node 1. In the configuration information matching the first effect editing, only preprocessing function node 1, deep learning inference node 2 used to transform the human face into a cartoon face, and postprocessing function node 3 are recorded, but the processing logic of preprocessing function node 1 is not recorded. Similarly, in the configuration information matching the second effect editing, only preprocessing function node 1, deep learning inference node 4 used to transform the human face into an animal face, and postprocessing function node 5 are recorded, but the processing logic of preprocessing function node 1 is not recorded. In this way, not only is the amount of configuration information matching the first type of special effect editing reduced, but the amount of configuration information matching the second type of special effect editing is also reduced. Furthermore, the processing logic of the pre-processing function node 1 is reused. During the development phase, the workload is reduced, which is conducive to improving development efficiency. Moreover, when maintaining the processing logic of the pre-processing function node 1, it is convenient, quick, and easy to implement, without the need to modify the configuration information for each special effect editor.

[0042] It should be noted that the post-processing function node 3 and post-processing function node 5 mentioned above may include the same post-processing function node, such as a node used for processing operations such as format conversion, size scaling, rotation or smoothing of the image.

[0043] In general, the preprocessing functional nodes associated with different deep learning inference nodes are the same; and / or, the postprocessing functional nodes associated with different deep learning inference nodes are the same. Correspondingly, refer to, for example... Figure 2 The diagram shown illustrates a processing chain for achieving different editing effects. Figure 2 As can be seen, the preprocessing function nodes associated with different deep learning inference nodes may be the same, such as the image transformation node, which is required in classification, segmentation and key point detection algorithms.

[0044] Similarly, the post-processing function nodes associated with different deep learning inference nodes may be the same. For example, both segmentation algorithms and key point detection algorithms require a temporal smoothing node. The processing logic corresponding to the temporal smoothing node can be used to filter the data output by the deep learning inference node to eliminate some of the noise.

[0045] Step 120, according to the configuration information, calling the processing logic of the pre-processing function node to transform the original image through the processing logic of the pre-processing function node, and obtaining input data conforming to the input requirement of the algorithm model corresponding to the deep learning inference node.

[0046] Taking the algorithm model corresponding to the deep learning inference node as an example, which is a model for transforming a human face into a cartoon face, the input data of the model is the data of the region where the human face is located in the original image, and the data needs to be float type data. Therefore, the data of the original image needs to be processed into the data of the region where the human face is located in the float type through the pre-processing function node. Specifically, the face in the original image can be cropped through an image cropping node (which can be understood as one of the pre-processing function nodes) to obtain a local image including only the face, and the face is calibrated to the center position of the local image. Then, the data type of the local image is converted into a float type through an image transformation node (which can be understood as one of the pre-processing function nodes) to obtain input data conforming to the input requirement of the algorithm model corresponding to the deep learning inference node.

[0047] Optionally, when the pre-processing function node includes multiple nodes, the configuration information further includes the execution order between the multiple nodes. For example, the image cropping node and the image transformation node in the above description both belong to the pre-processing function node, but the original image needs to be processed through the processing logic of the image cropping node first, and then processed through the processing logic of the image transformation node, that is, the execution order of the image cropping node is earlier than that of the image transformation node, and the execution order is identified through data in the configuration information.

[0048] Illustratively, according to the configuration information, calling the processing logic of the pre-processing function node includes sequentially calling the processing logic of multiple nodes in the pre-processing function node according to the execution order.

[0049] Similarly, when the post-processing function node includes multiple nodes, the configuration information further includes the execution order between the multiple nodes.

[0050] According to the configuration information, calling the processing logic of the post-processing function node includes sequentially calling the processing logic of multiple nodes in the post-processing function node according to the execution order.

[0051] Illustratively, the pre-processing function node includes one or more of the following nodes:

[0052] The image transformation node, the region detection node, and the region image cropping node. The processing logic corresponding to the image transformation node can be logic for preprocessing the original image, such as converting the data type of the original image, obtaining an image of a specified size by scaling the original image, or performing some noise reduction processing on the original image, and the like. The processing logic corresponding to the region detection node can be logic for determining the position of a target object (such as a face, a mouth, a nose, or an eye, or the like) in the original image. The processing logic corresponding to the region image cropping node can be logic for cutting out the target object from the original image.

[0053] The post-processing function node includes one or more of the following nodes:

[0054] The image transformation node and the time domain smoothing node.

[0055] Step 130, performing special effect editing on the original image based on the input data by the algorithm model corresponding to the deep learning inference node to obtain output data.

[0056] Step 140, calling the processing logic of the post-processing function node according to the configuration information to perform transformation processing on the output data by the processing logic of the post-processing function node to obtain a target image with added special effects.

[0057] The image processing method provided by the embodiments of the present disclosure sets configuration information matched with each special effect editing to only include a deep learning inference node for performing the special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node, without including the algorithm model corresponding to the deep learning inference node and the processing logic of the pre-processing function node and the post-processing function node. The algorithm model corresponding to the deep learning inference node and the processing logic of the pre-processing function node and the post-processing function node are stored separately from the configuration information, so that the same processing logic involved in different special effect editing can be reused, thereby reducing the data amount of the configuration information matched with each special effect editing, and further reducing the overall data amount of the application program for image processing and improving the development efficiency.

[0058] Figure 3 FIG. 1 is a structural schematic diagram of an image processing device in the embodiments of the present disclosure. The image processing device provided by the embodiments of the present disclosure can be configured in a client. The image processing device specifically includes an acquisition module 310, a first calling module 320, a second calling module 330, and a third calling module 340.

[0059] The obtaining module 310 is configured to obtain configuration information matched with special effect editing when the special effect editing is performed on an original image, the configuration information including a deep learning inference node used for performing the special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node; the first calling module 320 is configured to call processing logic of the pre-processing function node according to the configuration information, so as to perform transformation processing on the original image through the processing logic of the pre-processing function node, and obtain input data meeting input requirements of an algorithm model corresponding to the deep learning inference node; the second calling module 330 is configured to perform the special effect editing on the original image through the algorithm model corresponding to the deep learning inference node based on the input data, and obtain output data; and the third calling module 340 is configured to call processing logic of the post-processing function node according to the configuration information, so as to perform transformation processing on the output data through the processing logic of the post-processing function node, and obtain a target image to which the special effect is added.

[0060] Optionally, the pre-processing function nodes associated with different deep learning inference nodes are the same; and / or the post-processing function nodes associated with different deep learning inference nodes are the same.

[0061] Optionally, the pre-processing function node includes a plurality of nodes, and the configuration information further includes an execution order between the plurality of nodes.

[0062] Optionally, the first calling module 320 is specifically configured to sequentially call processing logic of the plurality of nodes in the pre-processing function node according to the execution order.

[0063] Optionally, the post-processing function node includes a plurality of nodes, and the configuration information further includes an execution order between the plurality of nodes.

[0064] Optionally, the second calling module 330 is specifically configured to sequentially call processing logic of the plurality of nodes in the post-processing function node according to the execution order.

[0065] Optionally, the pre-processing function node includes one or more of the following nodes:

[0066] an image transformation node, a region detection node, and a region image clipping node;

[0067] The post-processing function node includes one or more of the following nodes:

[0068] an image transformation node and a time domain smoothing node.

[0069] Optionally, the acquisition module 310 is specifically used to: determine a configuration file that has a preset binding relationship with the special effects editor; and read the configuration information from the configuration file.

[0070] The image processing apparatus provided in this disclosure embodiment can execute the steps performed by the client in the image processing method provided in this disclosure method embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0071] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 4 The diagram illustrates a structural schematic suitable for implementing the electronic device 500 in the embodiments of this disclosure. The electronic device 500 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0072] like Figure 4 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503 to implement the image processing method as described in the embodiments of this disclosure. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0073] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0074] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts, thereby implementing the image processing method as described above. In such embodiments, the computer program can be downloaded and installed from a network by the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0075] It should be noted that the computer-readable medium described above in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, an optical fiber, an RF (radio frequency) or the like, or any suitable combination thereof.

[0076] In some embodiments, the client, server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

[0077] The computer readable medium described above can be included in the electronic device described above; or can exist separately, without being assembled into the electronic device.

[0078] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to: when performing special effect editing on an original image, obtain configuration information matched with the special effect editing, the configuration information including a deep learning inference node used for performing special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node; according to the configuration information, call processing logic of the pre-processing function node to perform transformation processing on the original image through the processing logic of the pre-processing function node, to obtain input data meeting input requirements of an algorithm model corresponding to the deep learning inference node; perform special effect editing on the original image through the algorithm model corresponding to the deep learning inference node based on the input data, to obtain output data; according to the configuration information, call processing logic of the post-processing function node to perform transformation processing on the output data through the processing logic of the post-processing function node, to obtain a target image to which special effects are added.

[0079] Optionally, when the one or more programs are executed by the electronic device, the electronic device can further perform other steps described in the above embodiments.

[0080] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0081] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0082] The units described in the embodiments of the present disclosure can be implemented by hardware, software, or a combination of hardware and software. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0083] The functions described in this specification can be implemented in part or in whole through one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0084] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0085] According to one or more embodiments of the present disclosure, the present disclosure provides an image processing method, comprising: when performing special effect editing on an original image, obtaining configuration information matched with the special effect editing, the configuration information comprising a deep learning inference node used for performing special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node; calling processing logic of the pre-processing function node according to the configuration information, to perform transformation processing on the original image through the processing logic of the pre-processing function node, to obtain input data meeting input requirements of an algorithm model corresponding to the deep learning inference node; performing special effect editing on the original image based on the input data through the algorithm model corresponding to the deep learning inference node, to obtain output data; calling processing logic of the post-processing function node according to the configuration information, to perform transformation processing on the output data through the processing logic of the post-processing function node, to obtain a target image with added special effects.

[0086] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the pre-processing function nodes associated with different deep learning inference nodes are the same; and / or the post-processing function nodes associated with different deep learning inference nodes are the same.

[0087] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the pre-processing function node comprises a plurality of nodes, and the configuration information further comprises an execution order between the plurality of nodes.

[0088] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the calling of the processing logic of the pre-processing function node according to the configuration information comprises: sequentially calling the processing logic of multiple nodes in the pre-processing function node according to the execution order.

[0089] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the post-processing function node comprises multiple nodes, and the configuration information further comprises an execution order between the multiple nodes.

[0090] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the calling of the processing logic of the post-processing function node according to the configuration information comprises: sequentially calling the processing logic of multiple nodes in the post-processing function node according to the execution order.

[0091] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the pre-processing function node comprises one or more of the following nodes: an image transformation node, a region detection node, and a region image cropping node; and the post-processing function node comprises one or more of the following nodes: an image transformation node and a time domain smoothing node.

[0092] According to one or more embodiments of the present disclosure, in the image processing method provided by the present disclosure, optionally, the obtaining of the configuration information matched with the special effect editing comprises: determining a configuration file having a preset binding relationship with the special effect editing; and reading the configuration information from the configuration file.

[0093] According to one or more embodiments of the present disclosure, the present disclosure provides an image processing device, comprising: an obtaining module, configured to, when a special effect editing is performed on an original image, obtain configuration information matched with the special effect editing, the configuration information comprising a deep learning inference node used for performing the special effect editing on the original image, and a pre-processing function node and a post-processing function node associated with the deep learning inference node; a first calling module, configured to call processing logic of the pre-processing function node according to the configuration information, so as to perform transformation processing on the original image through the processing logic of the pre-processing function node, and obtain input data meeting input requirements of an algorithm model corresponding to the deep learning inference node; a second calling module, configured to perform the special effect editing on the original image based on the input data through the algorithm model corresponding to the deep learning inference node, and obtain output data; and a third calling module, configured to call processing logic of the post-processing function node according to the configuration information, so as to perform transformation processing on the output data through the processing logic of the post-processing function node, and obtain a target image to which a special effect is added.

[0094] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, comprising:

[0095] one or more processors;

[0096] a memory for storing one or more programs;

[0097] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of the present disclosure.

[0098] According to one or more embodiments of the present disclosure, the present disclosure provides a computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the image processing method according to any one of the present disclosure.

[0099] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology employed. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also encompass other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0100] In addition, although each operation is depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0101] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An image processing method, characterized in that, The method includes: When performing special effects editing on the original image, configuration information matching the special effects editing is obtained. The configuration information includes a deep learning inference node for performing special effects editing on the original image, as well as a pre-processing function node and a post-processing function node associated with the deep learning inference node. The processing logic of the preprocessing function node is invoked according to the configuration information to transform the original image through the processing logic of the preprocessing function node, so as to obtain input data that meets the input requirements of the algorithm model corresponding to the deep learning inference node. The input data is a local image of the face converted into floating-point data. The algorithm model corresponding to the deep learning inference node performs special effects editing on the original image based on the input data to obtain output data; The processing logic of the post-processing function node is invoked according to the configuration information, so as to transform the output data through the processing logic of the post-processing function node to obtain a target image with added special effects. The preprocessing function node includes one or more of the following nodes: Image transformation node, region detection node, and region image cropping node; The post-processing function node includes one or more of the following nodes: Image transformation node and temporal smoothing node.

2. The method according to claim 1, characterized in that, The preprocessing functional nodes associated with the different deep learning inference nodes are the same; And / or, the same post-processing functional node associated with different deep learning inference nodes.

3. The method according to claim 1, characterized in that, The preprocessing function node includes multiple nodes, and the configuration information also includes the execution order among the multiple nodes.

4. The method according to claim 3, characterized in that, The processing logic of the preprocessing function node is invoked according to the configuration information, including: The processing logic of multiple nodes in the preprocessing function node is called sequentially according to the execution order.

5. The method according to claim 1, characterized in that, The post-processing function node includes multiple nodes, and the configuration information also includes the execution order among the multiple nodes.

6. The method according to claim 5, characterized in that, The processing logic of the post-processing function node is invoked according to the configuration information, including: The processing logic of multiple nodes in the post-processing function node is called sequentially according to the execution order.

7. The method according to claim 1, characterized in that, The step of obtaining configuration information that matches the special effects editing includes: Identify configuration files that have a preset binding relationship with the special effects editor; Read the configuration information from the configuration file.

8. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire configuration information matching the special effects editing when performing special effects editing on the original image. The configuration information includes a deep learning inference node for performing special effects editing on the original image, as well as a pre-processing function node and a post-processing function node associated with the deep learning inference node. The first calling module is used to call the processing logic of the preprocessing function node according to the configuration information, so as to transform the original image through the processing logic of the preprocessing function node to obtain input data that meets the input requirements of the algorithm model corresponding to the deep learning inference node, wherein the input data is a local image of a face converted into floating-point data. The second calling module is used to perform special effects editing on the original image based on the input data through the algorithm model corresponding to the deep learning inference node, and obtain output data. The third calling module is used to call the processing logic of the post-processing function node according to the configuration information, so as to transform the output data through the processing logic of the post-processing function node to obtain a target image with added special effects. The preprocessing function node includes one or more of the following nodes: Image transformation node, region detection node, and region image cropping node; The post-processing function node includes one or more of the following nodes: Image transformation node and temporal smoothing node.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images