Image processing method, device, computer readable medium and electronic device

By processing the images captured by the terminal device camera and generating dynamic images, the problem of insufficient image fun in the prior art is solved, and the user experience and stickiness of social platforms are improved.

CN110163932BActive Publication Date: 2024-11-19TENCENT DIGITAL TIANJIN
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201810763192.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-07-12
Publication Date
2024-11-19
Estimated Expiration
2038-07-12

AI Technical Summary

Technical Problem

The existing terminal device camera photography function is insufficient in terms of fun and cannot bring users enough entertainment experience.

Method used

By obtaining the pending image, determine the pending object, such as a human face, an animal face or a cartoon character face, and process the image according to the target animation, generate a multi-frame processed image, each frame contains the pending object, and finally synthesize the dynamics image. The processing steps include scaling, moving and rotating, adding text based on the expression detection results, and optimizing image clarity.

Benefits of technology

It increases the fun of the images, improves the user experience and stickiness to social platforms, and provides a more entertaining dynamic image output.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110163932B_ABST
    Figure CN110163932B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention provide an image processing method, device, computer-readable medium and electronic device. The image processing method includes: obtaining an image to be processed; determining an object to be processed in the image to be processed, wherein the object to be processed includes at least one of a human face, an animal face or a cartoon character face; selecting a target dynamic image to be applied to the image to be processed; processing the image to be processed according to the target dynamic image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed; synthesizing the multiple frames of processed images into a dynamic image. The technical solution of the embodiment of the present invention can process the image to be processed based on the image to be processed including the object to be processed and the selected target dynamic image, obtain multiple frames of processed images, and synthesize the processed multiple frames of images to generate a final dynamic image, thereby increasing the interest of the image to be processed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular to an image processing method, device, computer-readable medium and electronic device. Background Art

[0002] Nowadays, taking photos with the camera of a terminal device has become a common entertainment method for users. In existing terminal devices, the camera functions are increasing, such as Selfie, panoramic shooting, etc., but these functions cannot bring enough fun to users.

[0003] Therefore, a new image processing method, device, computer-readable medium and electronic device are needed.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide an image processing method, device, computer-readable medium and electronic device, thereby overcoming the problem of insufficient image interest existing in the related art at least to a certain extent.

[0006] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0007] According to one aspect of an embodiment of the present invention, there is provided an image processing method, comprising: acquiring an image to be processed; determining an object to be processed in the image to be processed, wherein the object to be processed includes at least one of a human face, an animal face or a cartoon character face; selecting a target dynamic image to be applied to the image to be processed; processing the image to be processed according to the target dynamic image to generate a plurality of frames of processed images, wherein each frame of the plurality of frames of processed images includes the object to be processed; and synthesizing the plurality of frames of processed images into a dynamic image.

[0008] In some embodiments of the present invention, based on the aforementioned scheme, the image to be processed is processed according to the target dynamic image to generate multiple frames of processed images, including: determining the maximum scaling ratio of the multiple frames of processed images according to the original size of the object to be processed and the original size of the image to be processed; determining the actual scaling ratio of each frame in the multiple frames of processed images according to the scaling mode and the total number of frames of the target dynamic image and the maximum scaling ratio.

[0009] In some embodiments of the present invention, based on the aforementioned scheme, the zoom mode is a uniform zoom mode; wherein, determining the actual zoom ratio of each frame in the multi-frame processed image according to the zoom mode and the total number of frames of the target animation and the maximum zoom ratio includes: determining the zoom ratio of each frame in the total number of frames according to the maximum zoom ratio and the total number of frames of the target animation; if the zoom ratio of each frame in the total number of frames is less than a zoom threshold, reducing the total number of frames to a point where the actual zoom ratio of each frame in the multi-frame processed image is not less than the zoom threshold.

[0010] In some embodiments of the present invention, based on the aforementioned scheme, the image to be processed is processed according to the target motion picture to generate multiple frames of processed images, and also includes: determining the total movement distance of the object to be processed according to the original position of the object to be processed and the target position in the image to be processed; determining the movement distance of each frame in the multiple frames of processed images according to the movement mode of the target motion picture, the actual scaling ratio of each frame in the multiple frames of processed images, and the total movement distance.

[0011] In some embodiments of the present invention, based on the above-mentioned solution, the movement mode includes any one or more of a straight line uniform speed movement mode, a straight line variable speed movement mode, a curved line uniform speed movement mode, and a curved line variable speed movement mode.

[0012] In some embodiments of the present invention, based on the aforementioned scheme, the image to be processed is processed according to the target dynamic image to generate multiple frames of processed images, including: determining the rotation angle of each frame in the multiple frames of processed images according to the total rotation angle of the target dynamic image.

[0013] In some embodiments of the present invention, based on the aforementioned scheme, it also includes: based on at least one frame of the multiple frames of processed images, obtaining the expression detection result of the object to be processed; adding corresponding text to at least one frame of the multiple frames of processed images according to the expression detection result, generating at least one frame of processed image including corresponding text; wherein, synthesizing the multiple frames of processed images into a dynamic image includes: synthesizing the multiple frames of processed images and the at least one frame of processed image including corresponding text into the dynamic image.

[0014] In some embodiments of the present invention, based on the aforementioned scheme, the selection of the target animated picture to be applied to the image to be processed includes: determining the target animated picture to be applied to the image to be processed in response to an operation on the target animated picture; or recommending the target animated picture to be applied to the image to be processed based on the expression detection result.

[0015] In some embodiments of the present invention, based on the aforementioned scheme, there is at least one target motion picture; processing the image to be processed according to the target motion picture to generate multiple frames of processed images includes: if there are multiple target motion pictures, processing the image to be processed respectively according to the multiple target motion pictures to generate multiple frames of processed images respectively; wherein, synthesizing the multiple frames of processed images into a dynamic image includes: synthesizing the multiple frames of processed images generated according to each target motion picture into a dynamic image respectively; and sequentially splicing the dynamic images synthesized according to each target motion picture.

[0016] In some embodiments of the present invention, based on the above-mentioned solution, it also includes: evaluating the clarity of the image to be processed to obtain an evaluation result; and optimizing the clarity of the image to be processed according to the evaluation result.

[0017] In some embodiments of the present invention, based on the aforementioned scheme, optimizing the clarity of the image to be processed according to the evaluation result includes: optimizing the clarity of the image to be processed by using a super-resolution technology based on deep learning.

[0018] In some embodiments of the present invention, based on the aforementioned scheme, determining the object to be processed in the image to be processed includes: identifying the object in the image to be processed and obtaining the number of objects in the image to be processed; if the number of objects is equal to one, taking the object in the image to be processed as the object to be processed; if the number of objects is greater than one, determining the object to be processed in response to an operation on the image to be processed.

[0019] According to one aspect of an embodiment of the present invention, there is provided an image processing device, comprising: a module for acquiring an image to be processed, configured to acquire an image to be processed; a module for determining an object to be processed, configured to determine an object to be processed in the image to be processed; a target motion picture selection module, configured to select a target motion picture to be applied to the image to be processed; an image processing module, configured to process the image to be processed according to the target motion picture to generate a plurality of frames of processed images, wherein each frame of the plurality of frames of processed images includes the object to be processed; and an image synthesis module, configured to synthesize the plurality of frames of processed images into a dynamic image.

[0020] According to one aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the image processing method as described in the above embodiment is implemented.

[0021] According to one aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; and a storage device configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in the above embodiments.

[0022] In the technical solutions provided in some embodiments of the present invention, an image to be processed is obtained and an object to be processed in the image to be processed and a target dynamic image applied to the image to be processed are determined; the image to be processed is then processed according to the target dynamic image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed; and then the multiple frames of processed images are synthesized into a dynamic image, thereby increasing the fun of the image to be processed, improving user experience, and increasing user stickiness to the social platform.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into and constitute a part of the specification, showing embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 A schematic diagram showing an exemplary system architecture to which an image processing method or an image processing device according to an embodiment of the present invention can be applied;

[0026] Figure 2 A schematic diagram showing the structure of a computer system of an electronic device suitable for implementing an embodiment of the present invention is shown;

[0027] Figure 3 A flowchart of an image processing method according to an embodiment of the present invention is schematically shown;

[0028] Figure 4 Schematically shows Figure 3 A flowchart of an implementation process of step S320 shown in FIG.

[0029] Figure 5 Schematically shows Figure 3 A flowchart of an implementation process of step S340 shown in FIG.

[0030] Figure 6 Schematically shows Figure 5A flowchart of an implementation process of step S342 shown in FIG.

[0031] Figure 7 Schematically shows Figure 3 A flowchart of another implementation process of step S340 shown in FIG.

[0032] Figure 8 Schematically shows Figure 3 A flowchart of another implementation process of step S340 shown in FIG.

[0033] Fig. 9 A flowchart of an image processing method according to another embodiment of the present invention is schematically shown;

[0034] Fig.10 A flowchart of an image processing method according to another embodiment of the present invention is schematically shown;

[0035] Fig.11 A flowchart of an image processing method according to another embodiment of the present invention is schematically shown;

[0036] Fig.12 A schematic diagram schematically shows the display effect of an image processing method according to an embodiment of the present invention;

[0037] Fig.13 A block diagram schematically shows an image processing device according to another embodiment of the present invention;

[0038] Fig.14 Schematically shows Fig.13 A block diagram of an embodiment of an image processing module 1340 shown in FIG.

[0039] Fig.15 A block diagram of an image processing apparatus according to another embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0040] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0041] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0044] Figure 1 The diagram shows an exemplary system architecture 100 to which the image processing method or the image processing apparatus according to the embodiment of the present invention can be applied.

[0045] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0046] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0047] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, portable computers and desktop computers, intelligent chat robots, etc.

[0048] The server 105 may be a server that provides various services. For example, a user uses the terminal device 103 (or the terminal device 101 or 102) to send an image processing request to the server 105. The server 105 may process the image to be processed based on the image to be processed carried in the image processing request, the object to be processed in the image to be processed, and the target dynamic image selected by the user, generate multiple frames of processed images, synthesize the multiple frames of processed images into a dynamic image, and send the synthesized dynamic image to the terminal device 103, so that the user can determine the relationship between different entities based on the content displayed on the terminal device 103 and preview the synthesized dynamic image.

[0049] Figure 2 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention is shown.

[0050] It should be noted that Figure 2 The computer system 200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0051] like Figure 2 As shown, the computer system 200 includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 202 or a program loaded from a storage part 208 into a random access memory (RAM) 203. Various programs and data required for system operation are also stored in the RAM 203. The CPU 201, the ROM 202, and the RAM 203 are connected to each other via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0052] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, etc.; an output section 207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed, so that a computer program read therefrom is installed into the storage section 208 as needed.

[0053] In particular, according to an embodiment of the present invention, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 209, and / or installed from a removable medium 211. When the computer program is executed by the central processing unit (CPU) 201, various functions defined in the method and / or device of the present application are executed.

[0054] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0055] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods, devices and computer program products according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of boxes in the block diagram or flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0056] The modules and / or submodules and / or units and / or subunits involved in the embodiments of the present invention may be implemented in software or hardware, and the modules and / or submodules and / or units and / or subunits described may also be arranged in a processor. The names of these modules and / or submodules and / or units and / or subunits do not, in certain circumstances, constitute limitations on the modules and / or submodules and / or units and / or subunits themselves.

[0057] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being installed in the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiment. For example, the electronic device may implement the following method. Figure 3 or Figure 4 or Figure 6 or Figure 7 or Figure 8 or Fig. 9 or Fig.10 or Fig.11 The steps shown.

[0058] Figure 3 The flowchart of the image processing method according to an embodiment of the present invention is schematically shown.

[0059] like Figure 3 As shown, the image processing method provided by the embodiment of the present invention may include the following steps.

[0060] In step S310, an image to be processed is obtained.

[0061] In an exemplary embodiment, the acquiring of the image to be processed may include: determining the image to be processed in response to an operation on an image list.

[0062] For example, the image list may be a list of photos stored locally on the user's mobile terminal. For another example, the image list may also be a list of photos or pictures stored on a remote server that the user searches through the network. The present invention is not limited to this.

[0063] In step S320, an object to be processed in the image to be processed is determined, wherein the object to be processed includes at least one of a human face, an animal face or a cartoon character face.

[0064] For example, the object to be processed may be a human face in the image to be processed. For another example, the object to be processed may also be any one or more of animals such as a cat, a dog, etc. in the image to be processed. In the following embodiments, the object to be processed is described as a human face, but the present invention is not limited thereto.

[0065] In step S330, a target animated image to be applied to the image to be processed is selected.

[0066] In an exemplary embodiment, the method may further include: pre-storing at least one dynamic image template. Each dynamic image template may be designed and configured with various parameters according to the visual effect that the designer wants to achieve, such as the total number of frames of the synthesized dynamic image, whether each frame of the dynamic image is rotated, and if it is set to rotate, the total rotation angle may be further set.

[0067] In some embodiments, a user may select one or more of the at least one animated image templates as the target animated image of the image to be processed. In other embodiments, the system may also identify the expression of the object to be processed and automatically recommend a corresponding animated image template as the target animated image based on the expression detection result.

[0068] In step S340, the image to be processed is processed according to the target dynamic image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed.

[0069] In an embodiment of the present invention, the image to be processed can be processed accordingly according to the target animated image. For example, each frame of the multiple frames of processed images can be generated by scaling the image to be processed, translating the object to be processed in the image to be processed, rotating the image to be processed, and cropping each frame of the image according to a set specified size.

[0070] In an embodiment of the present invention, if the target animated image magnifies the object to be processed frame by frame, when the object to be processed, such as a face, is magnified, the background of the image to be processed may be magnified in proportion to the face. However, the present invention is not limited thereto.

[0071] In the embodiment of the present invention, if the target animated image performs transformation operations such as translation and / or rotation on the object to be processed, these transformation operations such as translation and / or rotation may also be performed on the entire image to be processed. However, the present invention is not limited thereto.

[0072] In step S350, the multiple frames of processed images are synthesized into a dynamic image.

[0073] It should be noted that each step in the above-mentioned image processing method provided in the embodiment of the present invention can be performed by a terminal device (such as the above-mentioned Figure 1 The terminal devices 101, 102, 103 in the example may also be executed by a server (such as the above Figure 1 The server 105 in the embodiment may be executed, or it may be partially executed by the terminal device and partially executed by the server, and the terminal device and the server may interact through a network connection. The present invention is not limited to this, and it may be selected and adjusted according to the actual application scenario.

[0074] The image processing method provided by the embodiment of the present invention obtains an image to be processed and determines an object to be processed in the image to be processed and a target dynamic image applied to the image to be processed; then the image to be processed is processed according to the target dynamic image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed; then the multiple frames of processed images are synthesized into a dynamic image, thereby increasing the interest of the image to be processed, improving the user experience, and increasing the user's stickiness to the social platform.

[0075] Figure 4 Schematically shows Figure 3 A flowchart of an implementation process of step S320 is shown in FIG.

[0076] like Figure 4 As shown, in the embodiment of the present invention, the above step S320 may further include the following steps.

[0077] In step S321, objects in the image to be processed are identified, and the number of objects in the image to be processed is obtained.

[0078] The embodiment of the present invention is described by taking the object as a human face as an example. All faces in the image to be processed can be detected by a face recognition algorithm, and the number of faces in the image to be processed can be determined accordingly.

[0079] In step S322, determine whether the number of objects is equal to one; if the number of objects is equal to one, proceed to step S323; otherwise, jump to step S324.

[0080] In the embodiment of the present invention, it is determined whether the number of faces is equal to one.

[0081] In step S323, if the number of objects is equal to one, the object in the image to be processed is used as the object to be processed.

[0082] In the embodiment of the present invention, if the number of human faces in the image to be processed is one, the human faces in the image to be processed may be directly used as the human faces to be processed.

[0083] In step S324, continue to determine whether the number of objects is greater than one; if the number of objects is greater than one, proceed to step S325; otherwise, jump to step S326.

[0084] In the embodiment of the present invention, it may be further determined whether the number of faces in the image to be processed is greater than one.

[0085] In step S325, if the number of objects is greater than one, the objects to be processed are determined in response to the operation on the image to be processed.

[0086] In an embodiment of the present invention, if the number of faces in the image to be processed is greater than one, a prompt message may be sent to the user, prompting the user to select at least one face from two or more faces in the image to be processed as the face to be processed.

[0087] It should be noted that when the number of faces in the image to be processed is greater than one, the user is not limited to selecting only one face as the face to be processed. One or more faces in the image to be processed may be determined as the face to be processed according to the user's preference. The present invention is not limited to this. In the following embodiments, selection of one face as the face to be processed is used as an example for explanation.

[0088] In step S326, information indicating that the object does not exist in the image to be processed is sent.

[0089] In the embodiment of the present invention, if it is detected that no human face exists in the image to be processed, a prompt message may be sent to the user to prompt the user that no human face exists in the image to be processed, thereby terminating the current operation.

[0090] Figure 5 Schematically shows Figure 3 A flowchart of an implementation process of step S340 is shown in FIG.

[0091] like Figure 5 As shown, the above step S340 provided in the embodiment of the present invention may further include the following steps.

[0092] In step S341, the maximum scaling ratio of the multiple frames of processed images is determined according to the original size of the object to be processed and the original size of the image to be processed.

[0093] In the embodiment of the present invention, if the target animated image selected by the user has been scaled, the total maximum scaling ratio of the processed images may be determined according to the original size of the face to be processed and the original size of the image to be processed.

[0094] For example, through face detection, the original size of the face to be processed in the image to be processed can be obtained. Assuming that the width and height of the original size of the face to be processed are recorded as fWidth and fHeight respectively, and the width and height of the original size of the image to be processed are recorded as pWidth and pHeight respectively, the maximum scaling ratio scaleMax of the face to be processed can be calculated by the following formula:

[0095] scaleMax=min(pWidth / fWidth, pHeight / fHeight)-k (1)

[0096] In the above formula (1), min is a minimum value operation; k is a preset constant. For example, the default value of k can be set to 0.2, but the present invention is not limited to this, and the backend server can dynamically send configuration modifications. The specific value of K is related to the visual effect desired by the designer.

[0097] In step S342, the actual scaling ratio of each frame in the multi-frame processed image is determined according to the scaling mode and the total number of frames of the target animated image and the maximum scaling ratio.

[0098] In the embodiment of the present invention, the zoom mode may include any one of a uniform speed zoom mode, a variable speed zoom mode, and the like.

[0099] The uniform speed scaling mode means that the scaling ratio of each frame in the synthesized dynamic image is the same. The variable speed scaling mode means that the scaling ratio of each frame in the synthesized dynamic image is not exactly the same, for example, the scaling ratio of each frame can be increased or decreased in equal proportion.

[0100] Figure 6 Schematically shows Figure 5 A flowchart of an implementation process of step S342 is shown in FIG.

[0101] like Figure 6As shown, here, the zoom mode is taken as a uniform speed zoom mode as an example for explanation, and the above step S342 provided in the embodiment of the present invention may further include the following steps.

[0102] In step S3421, the scaling ratio of each frame in the total number of frames is determined according to the maximum scaling ratio and the total number of frames of the target animated image.

[0103] For example, assume that the total number of frames preset in the target animated image is M, where M is a positive integer greater than or equal to 2. When the zoom mode is a uniform zoom mode, the zoom ratio scaleframe of each frame in the total number of frames can be calculated by the following formula:

[0104] scaleframe=scaleMax / M (2)

[0105] In step S3422, determine whether the zoom ratio of each frame in the total number of frames of the target animated image is less than the zoom threshold s; if it is less than the zoom threshold s, enter step S3423; otherwise, jump to step S3424.

[0106] In an embodiment of the present invention, it is determined whether the scaling ratio scaleframe of each frame in the total number of frames of the target dynamic image is less than a preset scaling threshold s. If the scaling ratio scaleframe of each frame is less than the scaling threshold s, the number of frames Ftotal of the image in the final synthesized dynamic image and the actual scaling ratio scaleframe' of each frame can be adjusted accordingly. If the scaling ratio scaleframe of each frame is not less than the scaling threshold s, the total number of frames M of the target dynamic image can be directly used as the number of frames Ftotal of the image in the final synthesized dynamic image (here, it means not considering the case of adding text in the dynamic image, see the description below for details).

[0107] In the embodiment of the present invention, the default value of s can be set to 1.1, but the present invention is not limited thereto. The value of s is related to the visual effect desired by the designer.

[0108] In step S3423, if the scaling ratio of each frame in the total number of frames of the target animated image is less than the scaling threshold s, the total number of frames of the target animated image is reduced until the actual scaling ratio of each frame in the multi-frame processed image is not less than the scaling threshold s.

[0109] In step S3424, if the scaling ratio of each frame in the total number of frames of the target animated image is not less than the scaling threshold s, the total number of frames of the target animated image is equal to the number of frames in the multi-frame processed image.

[0110] In an embodiment of the present invention, the scaling ratio scaleframe of each frame is first calculated according to the total number of frames M of the target dynamic image. If the scaling ratio scaleframe of each frame is less than the scaling threshold s, the total number of frames M is reduced until the actual scaling ratio scaleframe' of each frame is not less than the scaling threshold s. The number of frames at this time is the number of frames of the dynamic face image, which is recorded as Ftotal; otherwise, the number of frames of the dynamic face image is equal to the total number of frames M of the target dynamic image.

[0111] It should be noted that in the uniform speed scaling mode, the actual scaling ratio scaleframe' of each frame in the dynamic image is equal; in the variable speed scaling mode, the scaling ratio scaleframe' of each frame in the dynamic image may not be equal, and the actual scaling ratio scaleframe' of each frame image can use the same scaling ratio scaleframe as that in the target dynamic image. At this time, the actual scaling ratio scaleframe' of each frame may not be restricted by the scaling threshold s, but no matter what the scaling mode is, the sum of the actual scaling ratios scaleframe' of all frames in the dynamic image shall not exceed the maximum scaling ratio scaleMax.

[0112] Figure 7 Schematically shows Figure 3 A flowchart of another implementation process of step S340 is shown in FIG.

[0113] like Figure 7 As shown, the above step S340 provided in the embodiment of the present invention may further include the following steps.

[0114] In step S343, the total moving distance of the object to be processed is determined according to the original position of the object to be processed and the target position in the image to be processed.

[0115] In the embodiment of the present invention, the original position of the object to be processed in the image to be processed can be determined by a face recognition algorithm. Here, the center position (x1, y1) of the face to be processed is used as the original position of the object to be processed, but the present invention is not limited thereto.

[0116] In the embodiment of the present invention, the target position in the image to be processed can be determined according to the configuration information of the target animated image. In the following embodiment, the target position is taken as the center position (x2, y2) of the image to be processed as an example for explanation, that is, by moving frame by frame, the face to be processed is finally moved to the center position of the image. However, the present invention is not limited to this, and the specific target position can be configured according to the actual visual effect to be achieved.

[0117] The total moving distance Dtotal of the object to be processed can be determined by the following formula:

[0118]

[0119] In step S344, the moving distance of each frame in the multi-frame processed image is determined according to the moving mode of the target animated image, the actual scaling ratio of each frame in the multi-frame processed image, and the total moving distance.

[0120] In an exemplary embodiment, the movement mode may include any one or more of a straight-line uniform-speed movement mode, a straight-line variable-speed movement mode, a curved uniform-speed movement mode, a curved variable-speed movement mode, and the like.

[0121] Among them, the straight uniform speed movement mode means that the face to be processed in each frame of the dynamic image moves toward the center of the image along a straight line trajectory, and the moving distance of each frame is the same. The straight variable speed movement mode means that the face to be processed in each frame of the dynamic image moves toward the center of the image along a straight line trajectory, and the moving distance of each frame is not exactly the same, for example, the moving distance can increase or decrease frame by frame. The curved uniform speed movement mode means that the face to be processed in each frame of the dynamic image moves toward the center of the image along a curved trajectory, and the moving distance of each frame is the same. The curved trajectory can be, for example, any suitable shape such as wavy, semicircular, fan-shaped, etc. The curved variable speed movement mode means that the face to be processed in each frame of the dynamic image moves toward the center of the image along a curved trajectory, and the moving distance of each frame is not exactly the same.

[0122] Taking the linear uniform speed moving mode as an example, without considering the actual scaling ratio of each frame, the moving distance dframe of each frame in the multi-frame processed image can be calculated by the following formula:

[0123] dframe=Dtotal / Ftotal (4)

[0124] It should be noted that, in actual situations, if the target animated image performs a scaling operation on each frame, when calculating the moving distance of each frame, it is also necessary to consider the actual scaling ratio of each frame and make appropriate adjustments to its moving distance.

[0125] Figure 8 Schematically shows Figure 3 A flowchart of another implementation process of step S340 is shown in FIG.

[0126] like Figure 8 As shown, the above step S340 provided in the embodiment of the present invention may further include the following steps.

[0127] In step S345, the rotation angle angrot of each frame in the multi-frame processed image is determined according to the total rotation angle Rtotal of the target animated image.

[0128] In an embodiment of the present invention, if a rotation mode is set in the target dynamic image, for example, the rotation mode may include any one of a uniform rotation mode, a variable speed rotation mode, etc. The uniform rotation mode means that the rotation angle of each frame in the dynamic image is the same, and the variable speed rotation mode means that the rotation angle of each frame in the dynamic image is not exactly the same, for example, the rotation angle increases or decreases frame by frame.

[0129] Here, taking the rotation mode as the uniform speed rotation mode as an example, the rotation angle angrot of each frame image can be calculated by the following formula:

[0130] angrot=Rtotal / Ftotal (5)

[0131] It should be noted that the target animated image may be set to include only any one of the zoom mode, the move mode, the rotation mode, etc., or may be set to include two or more of them.

[0132] Furthermore, the above embodiments are all descriptions of the case where no text-adapted image is included in the dynamic image.

[0133] Fig. 9 The flowchart of an image processing method according to another embodiment of the present invention is schematically shown.

[0134] like Fig. 9 As shown, the image processing method provided by the embodiment of the present invention may include the following steps.

[0135] In step S910, an image to be processed is obtained.

[0136] In step S920, an object to be processed in the image to be processed is determined, wherein the object to be processed includes at least one of a human face, an animal face or a cartoon character face.

[0137] In step S930, a target animated image to be applied to the image to be processed is selected.

[0138] In step S940, the image to be processed is processed according to the target dynamic image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed.

[0139] The above steps S910 to S940 can refer to the above Figure 3 Steps S310 to S340 in the illustrated embodiment will not be described in detail herein.

[0140] In step S950, based on at least one frame of the multiple frames of processed images, an expression detection result of the object to be processed is obtained.

[0141] In the embodiment of the present invention, the expression recognition of the face to be processed can be performed based on the last frame, i.e., the Ftotal frame, in the above-mentioned multi-frame processed image to obtain the expression detection result. However, the present invention is not limited thereto, and it can perform expression recognition based on any one or more frames in the multi-frame processed image or the image to be processed. Thus, the expression detection result of the face to be processed can be further matched with appropriate text.

[0142] In an exemplary embodiment, the selecting of the target animated picture to be applied to the image to be processed may include: determining the target animated picture to be applied to the image to be processed in response to an operation on the target animated picture; or recommending the target animated picture to be applied to the image to be processed based on the expression detection result.

[0143] For example, one or more of the target templates may be determined as the target animated image according to a selection operation of the user.

[0144] For another example, if the system recognizes that the expression of the face to be processed is a big smile, it will automatically match a GIF template that expresses a happy mood as the target GIF; or, if the system recognizes that the face to be processed is a crying expression, it will automatically match a GIF template that expresses a bad mood as the target GIF.

[0145] In step S960, corresponding text is added to at least one frame of the multiple frames of processed images according to the expression detection result to generate at least one frame of processed image including corresponding text (assuming L frames, where L is a positive integer greater than or equal to 1).

[0146] In the embodiment of the present invention, the system will automatically adapt the corresponding text according to the detected expression of the face to be processed. For example, if the expression of the face to be processed is identified as unhappy, the text that may be adapted is: blue and thin, mushroom. Of course, in other embodiments, the user can also manually input the text he wants to add to the image.

[0147] In an embodiment of the present invention, the adapted corresponding text can be added to the last frame, i.e., the Ftotal frame, in the multi-frame processed image, but the present invention is not limited thereto. The frame or frames to which the text is added can be set by the target animation or by the user.

[0148] In step S970, the multiple frames of processed images and the at least one frame of processed image including corresponding text are synthesized into the dynamic image.

[0149] In the embodiment of the present invention, the Ftotal frame processed image and the L frame image generated in the above step S940 and step S960 are synthesized into a dynamic image, such as GIF, and the final dynamic image may include Ftotal+L frames.

[0150] In an exemplary embodiment, there is at least one target animated image.

[0151] In an exemplary embodiment, the processing of the image to be processed according to the target animated image to generate multiple frames of processed images may include: if there are multiple target animated images, the image to be processed may be processed respectively according to the multiple target animated images to generate multiple frames of processed images respectively. Wherein, synthesizing the multiple frames of processed images into a dynamic image may include: synthesizing the multiple frames of processed images generated according to each target animated image into a dynamic image respectively; and sequentially splicing the dynamic images synthesized according to each target animated image. In this way, the faces to be processed may be processed accordingly according to multiple target animated images respectively, and the corresponding dynamic images may be synthesized respectively. Finally, the dynamic images may be sequentially spliced ​​to form a longer dynamic image, which may further increase the fun of the dynamic image.

[0152] In an exemplary embodiment, the method may further include: in response to a selection operation, publishing the generated dynamic image to a selected social platform.

[0153] For example, the user may choose to publish the dynamic image generated in S970 to QQ space to share with friends.

[0154] In an exemplary embodiment, the method may further include: after synthesizing the dynamic image, previewing the synthesized dynamic image.

[0155] In an exemplary embodiment, the method may further include: in response to a user's selection of a save operation, saving the dynamic image to a designated directory of a terminal device and / or a server, and prompting the user to jump to a publishing page of a corresponding social platform.

[0156] It should be noted that the adaptation of corresponding text to the image is not necessarily limited to the expression recognition method, and can also be added in any other suitable manner.

[0157] Fig.10 The following schematically shows a flow chart of an image processing method according to another embodiment of the present invention.

[0158] like Fig.10 As shown, the image processing method provided by the embodiment of the present invention is similar to the above Figure 3 and Fig. 9 Compared with the embodiment shown, the following steps may also be included.

[0159] In step S1010, the clarity of the image to be processed is evaluated to obtain an evaluation result.

[0160] In an exemplary embodiment, the performing clarity evaluation on the image to be processed may include: performing clarity evaluation on the image to be processed using a no-reference picture quality assessment algorithm.

[0161] In the quality evaluation of no-reference images, the clarity of the image is an important indicator to measure the quality of the image. It can correspond well to people's subjective feelings. The low clarity of the image shows that the image is blurry. For the application of no-reference image quality evaluation, the appropriate algorithm can be selected for evaluation according to the situation of the image to be processed.

[0162] For example, any one of the Brenner gradient function, the Tenengrad gradient function, the Laplacian gradient function, the grayscale variance product function, the variance function, the energy gradient function, the Vollath function, etc. may be used.

[0163] In step S1020, the clarity of the image to be processed is optimized according to the evaluation result.

[0164] In an exemplary embodiment, optimizing the clarity of the image to be processed according to the evaluation result may include: optimizing the clarity of the image to be processed by using a super-resolution technology based on deep learning.

[0165] Among them, the super-resolution technology based on deep learning can convert low-resolution images into high-resolution images using deep learning. Its effect can reach or even exceed the current super-resolution solution, while the speed is increased by about ten to one hundred times, and it can run on ordinary mobile devices. Moreover, aliasing artifacts can be avoided. In the embodiment of the present invention, the image clarity optimization uses a super-resolution technology based on deep learning to ensure that each frame in the dynamic image has a relatively high clarity.

[0166] In other embodiments, other methods may be used to optimize clarity, such as upsampling to reconstruct a low-resolution image into a larger image with more pixels and higher quality. For example, the upsampling method may be a linear method, that is, adding new pixel values ​​by performing a simple, fixed combination of known pixel values. Because a fixed linear filter is used (a constant convolution kernel processes the entire image indiscriminately), this method is very fast.

[0167] It should be noted that in the embodiment of the present invention, only the clarity of the image to be processed may be optimized. However, in other embodiments, each frame in the dynamic image may be first obtained, and then the clarity of each frame of the image may be optimized respectively.

[0168] Fig.11 A flowchart of an image processing method according to another embodiment of the present invention is schematically shown. Here, the specific process of the image processing method provided by the above embodiment is explained by taking the default animated image template as an example, wherein the default animated image template can be a face moving uniformly along a straight line toward the center of the image, and is uniformly enlarged frame by frame without other special effects such as rotation.

[0169] like Fig.11 As shown, the image processing method provided by the embodiment of the present invention may include the following steps.

[0170] In step S1101, entry.

[0171] In step S1102, the user selects a photo.

[0172] In the embodiment of the present invention, in order to generate an interesting dynamic image using a face photo, one can enter the photo list through the "Funny Face Motion Picture" entrance and select a photo containing the face photo.

[0173] In step S1103, face detection is performed.

[0174] In the embodiment of the present invention, after confirming the selected photo, the system automatically identifies the size, position and number of faces in the selected photo.

[0175] Specifically, any facial recognition algorithm may be used for face detection, such as a recognition algorithm based on facial feature points, a recognition algorithm based on a whole face image, a recognition algorithm based on a template, an algorithm using a neural network for recognition, and the like.

[0176] Among them, as the first step of face recognition, face detection is to detect the face from the image background. Detection is to determine whether there is a face in an image, and positioning is to give the position of the face in the image. The facial image information obtained after positioning is the pattern of the measurement space. To perform recognition work, the data in the measurement space must first be mapped to the feature space. For example, the principal component analysis method can be used. The principle is to project a high-dimensional vector into a low-dimensional vector space through a special eigenvector matrix, and represent it as a low-dimensional vector, and only lose some minor information. By performing feature extraction operations on the detected and positioned face images, the image dimension can be reduced, thereby reducing the recognition calculation amount and improving the recognition accuracy.

[0177] In step S1104, determine whether the number of faces detected in the photo is greater than or equal to 1; if the number of faces detected in the photo is greater than or equal to 1, go to step S1105; if the number of faces detected in the photo is less than 1, jump to step S1121.

[0178] In step S1105, continue to determine whether the number of faces detected in the photo is greater than 1; if the number of faces detected in the photo is greater than 1, proceed to step S1106; if the number of faces detected in the photo is equal to 1, jump to step S1107.

[0179] In step S1106, the user is asked to select one of them.

[0180] In the embodiment of the present invention, if multiple faces are detected in the selected photo, the user can select a face that is desired to be used to generate a dynamic image as a face to be processed.

[0181] In step S1107, a dynamic image template is selected as a target dynamic image.

[0182] In an embodiment of the present invention, the user can select a dynamic image template as the target dynamic image. The target dynamic image will affect the magnification of the face in each frame of the synthesized dynamic image, whether it moves at a constant speed, the movement trajectory toward the center of the image (straight line or curve), the rotation angle (0 means no rotation, 360 means the Ftotal frame rotates a total of 360 degrees), etc.

[0183] For example, the final dynamic image effect can be: the face is gradually enlarged, and the center of the face moves along a straight line toward the center of the image at a constant speed; or, the center of the face can move along a curve toward the center of the image at an uneven speed, and the image can be rotated in the process of moving toward the center of the image.

[0184] In step S1108, the total magnification factor is calculated according to the face size and the selected animation template.

[0185] In step S1109, the clarity is evaluated.

[0186] In step S1110, determine whether clarity optimization is required; if clarity optimization is required, proceed to step S1111; if clarity optimization is not required, jump to step S1112.

[0187] In step S1111, the image clarity is optimized based on super-resolution technology.

[0188] In step S1112, the distance between the center of the face and the center of the photo, the magnification of the face in each frame image, and other parameters are calculated.

[0189] In the embodiment of the present invention, the scaling ratio of each frame in the dynamic image, the distance to move to the center of the image, and the rotation angle can be calculated according to the original size and original position of the face to be processed and the original size of the selected image to be processed. For example, the trajectory to the center of the image can be a straight line or a curve, and different movement trajectories can achieve different interesting effects. The movement trajectory and rotation angle are related to the selected target dynamic image.

[0190] In step S1113, the actual magnification factor and the number of magnification times N of each frame are calculated by comprehensively considering the number of frames and the total magnification factor of the face, where N is an integer greater than or equal to 0.

[0191] In step S1114, determine whether N is greater than 0; if N is greater than 0, proceed to step S1115; if N is equal to 0, jump to step S1117.

[0192] In step S1115, the image is enlarged, and translated a certain distance toward the center of the image, and a frame of image data is generated by cropping to a specified size.

[0193] In the embodiment of the present invention, the specified size cropping means that certain restrictions can be placed on the size of the dynamic image. For example, the default policy is that the long side of the dynamic image does not exceed 960, but the present invention is not limited thereto, and dynamic adjustment of the configuration can be sent down.

[0194] In step S1116, N=N-1, and the process continues to jump back to step S1114, and the above steps S1114 to S1116 are executed in a loop until N is 0.

[0195] In step S1117, facial expression recognition.

[0196] In step S1118, based on the expression, the text description is adapted and one or more frames of image data are generated.

[0197] In step S1119, each frame of image generated in the above process is synthesized into a dynamic image.

[0198] In step S1120, after the user confirms, the dynamic image is published, and then the process jumps to step S1122 to end the operation.

[0199] In step S1121, it is prompted that no face is detected, and then jumps to step S1122.

[0200] In step S1122, end.

[0201] The image processing method provided by the embodiment of the present invention uses a photo containing a face to generate a dynamic image by gradually adjusting the position, size and rotation angle of the face, and can further adapt a suitable text based on the facial expression detection to further increase the interest of the dynamic image.

[0202] Using the above Fig.11 The image processing method of the embodiment shown in the figure, each frame of the final synthesized dynamic image has the following effect: Fig.12 shown.

[0203] Here, Ftotal is equal to 4, that is, the first four frames of images are the face enlarged frame by frame, the face center is gradually translated to the center of the image, and it is cropped to a specified size, so that the size of each frame is the same. L is equal to 3, that is, according to the facial expression detection, 3 sentences are adapted, "I am not targeting anyone" is added to the fifth image, "I am talking about everyone here" is added to the sixth image, and "All are retail investors" is added to the seventh image.

[0204] It should be noted that Fig.12 In the example shown, L lines of text are added one by one to L frames of images, but in other embodiments, L lines of text may also be added to the same frame of image, and the present invention is not limited to this.

[0205] The following describes an embodiment of the device of the present invention, which can be used to execute the above-mentioned image processing method of the present invention. For details not disclosed in the embodiment of the device of the present invention, please refer to the embodiment of the image processing method of the present invention.

[0206] Fig.13 A block diagram of an image processing apparatus according to another embodiment of the present invention is schematically shown.

[0207] like Fig.13 As shown, the image processing device 1300 provided in the embodiment of the present invention may include a to-be-processed image acquisition module 1310 , a to-be-processed object determination module 1320 , a target motion picture selection module 1330 , an image processing module 1340 and an image synthesis module 1350 .

[0208] The to-be-processed image acquisition module 1310 may be configured to acquire an to-be-processed image.

[0209] In an exemplary embodiment, the to-be-processed image acquisition module 1310 may further include an to-be-processed image acquisition unit. The to-be-processed image acquisition unit may be configured to determine the to-be-processed image in response to an operation on the image list.

[0210] The to-be-processed object determination module 1320 may be configured to determine the to-be-processed object in the to-be-processed image, wherein the to-be-processed object includes at least one of a human face, an animal face, or a cartoon character face.

[0211] In an exemplary embodiment, the object to be processed determination module 1320 may further include an object recognition unit, a first object to be processed determination unit, and a second object to be processed determination unit. The object recognition unit may be configured to recognize objects in the image to be processed and obtain the number of objects in the image to be processed. The first object to be processed determination unit may be configured to use the objects in the image to be processed as the objects to be processed if the number of objects is equal to one. The second object to be processed determination unit may be configured to determine the objects to be processed in response to an operation on the image to be processed if the number of objects is greater than one.

[0212] The target dynamic image selection module 1330 may be configured to select a target dynamic image to be applied to the image to be processed.

[0213] The image processing module 1340 may be configured to process the image to be processed according to the target moving image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed.

[0214] The image synthesis module 1350 may be configured to synthesize the multiple frames of processed images into a dynamic image.

[0215] In an exemplary embodiment, the image processing apparatus 1300 may further include a clarity evaluation module and a clarity optimization module.

[0216] The clarity evaluation module may be configured to evaluate the clarity of the image to be processed and obtain an evaluation result. The clarity optimization module may be configured to optimize the clarity of the image to be processed according to the evaluation result.

[0217] In an exemplary embodiment, the clarity evaluation module may further include a clarity evaluation unit. The clarity evaluation unit may be configured to use a non-reference picture quality assessment algorithm to perform clarity evaluation on the image to be processed.

[0218] In an exemplary embodiment, the definition optimization module may further include a definition optimization unit. The definition optimization unit may be configured to optimize the definition of the image to be processed by using a super-resolution method based on deep learning.

[0219] Fig.14 Schematically shows Fig.13 A block diagram of an embodiment of an image processing module 1340 is shown in FIG.

[0220] like Fig.14 As shown above Fig.13 The image processing module 1340 in the illustrated embodiment may further include a scaling submodule 1341 .

[0221] The scaling submodule 1341 may further include a maximum scaling ratio determining unit and an actual scaling ratio determining unit. The maximum scaling ratio determining unit may be configured to determine the maximum scaling ratio of the multiple processed images according to the original size of the object to be processed and the original size of the image to be processed. The actual scaling ratio determining unit may be configured to determine the actual scaling ratio of each frame in the multiple processed images according to the scaling mode and the total number of frames of the target animated image and the maximum scaling ratio.

[0222] In an exemplary embodiment, the zoom mode may be a uniform zoom mode.

[0223] In an exemplary embodiment, the actual scaling ratio determining unit may further include a template frame scaling ratio determining subunit and a dynamic frame scaling ratio determining subunit.

[0224] The template frame scaling ratio determination subunit may be configured to determine the scaling ratio of each frame in the total number of frames according to the maximum scaling ratio and the total number of frames of the target dynamic image. The dynamic frame scaling ratio determination subunit may be configured to reduce the total number of frames to an actual scaling ratio of each frame in the multi-frame processed image not less than the scaling threshold if the scaling ratio of each frame in the total number of frames is less than a scaling threshold.

[0225] Continue to refer Fig.14 , the above Fig.13 The image processing module 1340 in the illustrated embodiment may further include a movement submodule 1342 .

[0226] The moving submodule 1342 may further include a moving total distance determining unit and a frame moving distance determining unit.

[0227] The total moving distance determining unit may be configured to determine the total moving distance of the object to be processed according to the original position of the object to be processed and the target position in the image to be processed. The frame moving distance determining unit may be configured to determine the moving distance of each frame in the multi-frame processed image according to the moving mode of the target animated image, the actual scaling ratio of each frame in the multi-frame processed image, and the total moving distance.

[0228] In an exemplary embodiment, the movement mode includes any one or more of a straight-line uniform-speed movement mode, a straight-line variable-speed movement mode, a curved-line uniform-speed movement mode, and a curved-line variable-speed movement mode.

[0229] exist Fig.14 In the embodiment shown, the above Fig.13 The image processing module 1340 in the illustrated embodiment may further include a rotation submodule 1343 .

[0230] The rotation submodule 1343 may be configured to determine the rotation angle of each frame in the multiple processed images according to the total rotation angle of the target animated image.

[0231] Fig.15 A block diagram of an image processing device according to another embodiment of the present invention is schematically shown.

[0232] like Fig.15 As shown, the image processing device 1500 provided in an embodiment of the present invention may include a module 1510 for acquiring an image to be processed, a module 1520 for determining an object to be processed, a target animated image selection module 1530, an image processing module 1540, an expression detection module 1550, a text adding module 1560 and an image synthesis module 1570.

[0233] The to-be-processed image acquisition module 1510 may be configured to acquire the to-be-processed image.

[0234] The to-be-processed object determining module 1520 may be configured to determine the to-be-processed object in the to-be-processed image.

[0235] The target dynamic image selection module 1530 may be configured to determine a target dynamic image to be applied to the image to be processed.

[0236] The image processing module 1540 may be configured to process the image to be processed according to the target moving image to generate multiple frames of processed images, wherein each frame of the multiple frames of processed images includes the object to be processed.

[0237] The expression detection module 1550 may be configured to obtain an expression detection result of the object to be processed based on at least one frame of the multiple frames of processed images.

[0238] In an exemplary embodiment, the target animation selection module 1530 may further include: a target animation selection unit, which may be configured to determine a target animation to be applied to the image to be processed in response to an operation on the target animation; or a target animation recommendation unit, which may be configured to recommend a target animation to be applied to the image to be processed based on the expression detection result.

[0239] The text adding module 1560 may be configured to add corresponding text to at least one frame of the multiple frames of processed images according to the expression detection result, and generate at least one frame of processed image including the corresponding text.

[0240] The image synthesis module 1570 may be configured to synthesize the multiple frames of processed images and the at least one frame of processed image including corresponding text into the dynamic image.

[0241] In an exemplary embodiment, there is at least one target animated image.

[0242] In an exemplary embodiment, the image processing module 1540 may be configured to process the image to be processed according to the multiple target dynamic images respectively if there are multiple target dynamic images, and generate multiple frames of processed images respectively. The image synthesis module 1570 may further include: an image synthesis unit, which may be configured to synthesize the multiple frames of processed images generated according to each target dynamic image into a dynamic image; and a dynamic image splicing unit, which may be configured to sequentially splice the dynamic images synthesized according to each target dynamic image.

[0243] It should be noted that, although several modules or submodules or units or subunits of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or submodules or units or subunits described above can be concretized in one module or submodule or unit or subunit. On the contrary, the features and functions of one module or submodule or unit or subunit described above can be further divided into being concretized by multiple modules or submodules or units or subunits.

[0244] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0245] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0246] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Get the image to be processed; Determining an object to be processed in the image to be processed, wherein the object to be processed includes at least one of a human face, an animal face, or a cartoon character face; Selecting a target animated image to be applied to the image to be processed; Processing the image to be processed according to the target motion picture to generate multiple frames of processed images, which includes: determining the maximum scaling ratio of the multiple frames of processed images according to the original size of the object to be processed and the original size of the image to be processed; determining the actual scaling ratio of each frame in the multiple frames of processed images according to the scaling mode and total number of frames of the target motion picture and the maximum scaling ratio so that the background of the image to be processed is scaled frame by frame in proportion with the object to be processed; determining the total moving distance of the object to be processed according to the center position of the object to be processed and the center position of the image to be processed; determining the moving distance of each frame in the multiple frames of processed images according to the moving mode of the target motion picture, the actual scaling ratio of each frame in the multiple frames of processed images and the total moving distance, and moving the object to be processed to the center position of the image to be processed by moving frame by frame to generate the multiple frames of processed images; wherein each frame in the multiple frames of processed images includes the object to be processed; The multiple frames of processed images are synthesized into a dynamic image.

2. The method according to claim 1, characterized in that The zoom mode is a uniform zoom mode; wherein, determining the actual zoom ratio of each frame in the multi-frame processed image according to the zoom mode and the total number of frames of the target animated image and the maximum zoom ratio comprises: Determining a scaling ratio of each frame in the total number of frames according to the maximum scaling ratio and the total number of frames of the target animated image; If the scaling ratio of each frame in the total number of frames is less than the scaling threshold, the total number of frames is reduced until the actual scaling ratio of each frame in the multi-frame processed image is not less than the scaling threshold.

3. The method according to claim 1, characterized in that: The movement mode includes any one or more of a straight-line uniform-speed movement mode, a straight-line variable-speed movement mode, a curved uniform-speed movement mode, and a curved variable-speed movement mode.

4. The method according to any one of claims 1 to 3, characterized in that: The processing of the image to be processed according to the target dynamic image to generate multiple frames of processed images also includes: The rotation angle of each frame in the multiple frames of processed images is determined according to the total rotation angle of the target dynamic image to generate the multiple frames of processed images.

5. The method according to claim 1, characterized in that Also includes: Based on at least one frame of the multiple frames of processed images, obtaining an expression detection result of the object to be processed; Adding corresponding text to at least one frame of the multiple frames of processed images according to the expression detection result to generate at least one frame of processed image including the corresponding text; The step of synthesizing the multiple frames of processed images into a dynamic image includes: The multiple frames of processed images and the at least one frame of processed image including corresponding text are synthesized into the dynamic image.

6. The method according to claim 5, characterized in that The selecting a target animated image to be applied to the image to be processed comprises: In response to the operation on the target dynamic image, determining a target dynamic image to be applied to the image to be processed; or A target animated image applied to the image to be processed is recommended according to the expression detection result.

7. The method according to claim 1, characterized in that The target dynamic image is at least one; and the processing of the image to be processed according to the target dynamic image to generate multiple frames of processed images also includes: If there are multiple target motion pictures, the image to be processed is processed respectively according to the multiple target motion pictures to generate multiple frames of processed images respectively; The step of synthesizing the multiple processed images into a dynamic image comprises: Combining multiple frames of processed images generated according to each target dynamic image into a dynamic image; Sequentially stitch together dynamic images synthesized from each target animation.

8. The method according to claim 1, characterized in that Also includes: Performing clarity evaluation on the image to be processed to obtain an evaluation result; The clarity of the image to be processed is optimized according to the evaluation result.

9. The method according to claim 8, characterized in that Optimizing the clarity of the image to be processed according to the evaluation result includes: A super-resolution method based on deep learning is used to optimize the clarity of the image to be processed.

10. The method according to claim 1, characterized in that The determining the object to be processed in the image to be processed includes: Identify objects in the image to be processed, and obtain the number of objects in the image to be processed; If the number of objects is equal to one, the objects in the image to be processed are taken as the objects to be processed; If the number of objects is greater than one, the objects to be processed are determined in response to an operation on the image to be processed.

11. An image processing device, characterized in that: include: A module for acquiring an image to be processed, configured to acquire an image to be processed; a module for determining an object to be processed, configured to determine an object to be processed in the image to be processed, wherein the object to be processed includes at least one of a human face, an animal face or a cartoon character face; A target dynamic image selection module, configured to select a target dynamic image to be applied to the image to be processed; The image processing module is configured to process the image to be processed according to the target dynamic image to generate multiple frames of processed images. The image processing module includes a scaling submodule and a moving submodule. The scaling submodule includes: a maximum scaling ratio determining unit, configured to determine the maximum scaling ratio of the multiple frames of processed images according to the original size of the object to be processed and the original size of the image to be processed; an actual scaling ratio determining unit, configured to determine the actual scaling ratio of each frame in the multiple frames of processed images according to the scaling mode and total number of frames of the target dynamic image and the maximum scaling ratio so that the background of the image to be processed is proportional to the object to be processed. The example of frame-by-frame scaling; the moving submodule includes: a total moving distance determining unit, configured to determine the total moving distance of the object to be processed according to the center position of the object to be processed and the center position of the image to be processed; a frame moving distance determining unit, configured to determine the moving distance of each frame in the multi-frame processed image according to the moving mode of the target dynamic image, the actual scaling ratio of each frame in the multi-frame processed image and the total moving distance, and move the object to be processed to the center position of the image to be processed by moving frame by frame to generate the multi-frame processed image; wherein each frame in the multi-frame processed image includes the object to be processed; The image synthesis module is configured to synthesize the multiple frames of processed images into a dynamic image.

12. The device according to claim 11, characterized in that The scaling mode is a uniform scaling mode; wherein the actual scaling ratio determining unit comprises: a template frame scaling ratio determining subunit, configured to determine a scaling ratio of each frame in the total number of frames according to the maximum scaling ratio and the total number of frames of the target animated image; The dynamic frame scaling ratio determination subunit is configured to reduce the total number of frames until the actual scaling ratio of each frame in the multi-frame processed image is not less than the scaling threshold if the scaling ratio of each frame in the total number of frames is less than the scaling threshold.

13. The device according to claim 11, characterized in that The movement mode includes any one or more of a straight-line uniform-speed movement mode, a straight-line variable-speed movement mode, a curved uniform-speed movement mode, and a curved variable-speed movement mode.

14. The device according to any one of claims 11 to 13, characterized in that The image processing module also includes: The rotation submodule is configured to determine the rotation angle of each frame in the multiple frames of processed images according to the total rotation angle of the target dynamic image to generate the multiple frames of processed images.

15. The device according to claim 11, characterized in that Also includes: An expression detection module, configured to obtain an expression detection result of the object to be processed based on at least one frame of the multiple frames of processed images; A text adding module, configured to add corresponding text to at least one frame of the multiple frames of processed images according to the expression detection result, to generate at least one frame of processed image including the corresponding text; Wherein, the image synthesis module is configured to synthesize the multiple frames of processed images and the at least one frame of processed image including corresponding text into the dynamic image.

16. The device according to claim 15, characterized in that The target dynamic image selection module includes: a target dynamic image selection unit, configured to determine a target dynamic image to be applied to the image to be processed in response to an operation on the target dynamic image; or The target animated picture recommendation unit is configured to recommend a target animated picture to be applied to the image to be processed according to the expression detection result.

17. The device according to claim 11, characterized in that There is at least one target motion picture; the image processing module is configured to: if there are multiple target motion pictures, process the to-be-processed images respectively according to the multiple target motion pictures, and generate multiple frames of processed images respectively; Wherein, the image synthesis module includes: An image synthesis unit configured to synthesize multiple frames of processed images generated according to each target dynamic image into a dynamic image; The dynamic image stitching unit is configured to sequentially stitch the dynamic images synthesized according to the target dynamic images.

18. The device according to claim 11, characterized in that Also includes: A clarity evaluation module, configured to evaluate the clarity of the image to be processed and obtain an evaluation result; The clarity optimization module is configured to optimize the clarity of the image to be processed according to the evaluation result.

19. The device according to claim 18, characterized in that The clarity optimization module comprises: The clarity optimization unit is configured to optimize the clarity of the image to be processed by adopting a super-resolution method based on deep learning.

20. The device according to claim 11, characterized in that The module for determining the object to be processed comprises: an object recognition unit, configured to recognize objects in the image to be processed and obtain the number of objects in the image to be processed; a first to-be-processed object determination unit, configured to, if the number of objects is equal to one, take the object in the to-be-processed image as the to-be-processed object; The second to-be-processed object determination unit is configured to determine the to-be-processed object in response to an operation on the to-be-processed image if the number of the objects is greater than one.

21. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image processing method according to any one of claims 1 to 10 is implemented.

22. An electronic device, characterized in that: include: one or more processors; A storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1 to 10.

23. A computer program product, characterized in that It comprises a computer program, which contains a program code for executing the image processing method according to any one of claims 1 to 10.