Image processing method, device, intelligent terminal and storage medium

Abstract

The embodiment of the present invention discloses an image processing method, device, intelligent terminal and storage medium, wherein the method comprises: performing key point recognition on a target object in a target image to be processed to obtain a key point set of the target object, wherein the key point set comprises multiple image key points; obtaining target sub-material data associated with the target image from material data, wherein the target sub-material data comprises: a rendered image and rendering part indication information of the rendered image; determining a key point subset from the key point set according to the rendering part indication information, and determining the image position of each image key point in the key point subset in the target image; determining rendering parameters for the rendered image at the image position of the target image according to each image key point, and adding the rendered image to the target image according to the rendering parameters, which can improve the accuracy of image rendering and thus improve the rendering effect.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device, intelligent terminal and storage medium. Background Art

[0002] With the continuous development of image capture technology, more and more applications provide image rendering functions in the image capture process, that is, rendering a specified image into the captured image to be processed to enrich the image content of the image to be processed, such as rendering a specified heart pattern on the face of the image to be processed, thereby beautifying the image to be processed. Currently, when rendering the image to be processed, it is necessary to partition the image to be processed. The human-scene segmentation method is often used to distinguish the human area and non-human area in the image to be processed, and the specific parts of the human cannot be distinguished in detail. Therefore, the current image rendering function has the problem of low rendering position positioning accuracy. Therefore, how to improve the accuracy of the rendering process is a current research hotspot. Summary of the invention

[0003] The embodiments of the present invention provide an image processing method, device, server and storage medium, which can improve the accuracy of image rendering, thereby improving the rendering effect.

[0004] In one aspect, an embodiment of the present invention provides an image processing method, the method comprising:

[0005] Performing key point recognition on a target object in a target image to be processed to obtain a key point set of the target object, wherein the key point set includes a plurality of image key points;

[0006] Acquire target sub-material data associated with the target image from the material data, wherein the target sub-material data includes: a rendered image and rendering part indication information of the rendered image;

[0007] Determine a key point subset from the key point set according to the rendering part indication information, and determine the image position of each image key point in the key point subset in the target image;

[0008] Determine rendering parameters for the rendered image at the image position of the target image according to each image key point, and add the rendered image to the target image according to the rendering parameters.

[0009] On the other hand, an embodiment of the present invention provides an image processing device, the device comprising:

[0010] An identification unit, configured to perform key point identification on a target object in a target image to be processed, and obtain a key point set of the target object, wherein the key point set includes a plurality of image key points;

[0011] An acquisition unit, configured to acquire target sub-material data associated with the target image from the material data, wherein the target sub-material data includes: a rendered image and rendering part indication information of the rendered image;

[0012] a determining unit, configured to determine a key point subset from the key point set according to the rendering part indication information, and determine an image position of each image key point in the key point subset in the target image;

[0013] A processing unit is used to determine rendering parameters for the rendered image according to the image positions of the image key points in the target image, and add the rendered image to the target image according to the rendering parameters.

[0014] In another aspect, an embodiment of the present invention provides an intelligent terminal, including a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program that supports the terminal to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the following steps:

[0015] Performing key point recognition on a target object in a target image to be processed to obtain a key point set of the target object, wherein the key point set includes a plurality of image key points;

[0016] Acquire target sub-material data associated with the target image from the material data, wherein the target sub-material data includes: a rendered image and rendering part indication information of the rendered image;

[0017] Determine a key point subset from the key point set according to the rendering part indication information, and determine the image position of each image key point in the key point subset in the target image;

[0018] Determine rendering parameters for the rendered image at the image position of the target image according to each image key point, and add the rendered image to the target image according to the rendering parameters.

[0019] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, they are used to execute the image processing method as described in the first aspect.

[0020] In an embodiment of the present invention, after determining the target image to be processed, the intelligent terminal can perform key point recognition on the target object in the target image to obtain a key point set of the target object. After acquiring the target sub-material data associated with the target image from the material data, a key point subset can be determined from the key point set according to the rendering part indication information included in the target sub-material data, so that rendering parameters can be determined for the rendering image included in the target sub-material data according to the image position of each image key point in the key point subset in the target image, and the rendering image can be added to the target image according to the rendering parameters, so that the determined rendering position is more accurate, which is conducive to better fitting the rendering image and the target object and improving the rendering effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0022] Figure 1 is a schematic diagram of identifying key points of a target object provided by an embodiment of the present invention;

[0023] Figure 2 is a schematic flow chart of an image processing method provided by an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of material data provided by an embodiment of the present invention;

[0025] Figure 4 is a schematic flow chart of an image processing method provided by another embodiment of the present invention;

[0026] Figure 5a is a schematic diagram of determining a first angle according to an image key point provided by an embodiment of the present invention;

[0027] Figure 5b is a schematic diagram of a second angle of a rendered image provided by an embodiment of the present invention;

[0028] Figure 6a It is a schematic diagram of rendering a target image when there is one key reference point provided by an embodiment of the present invention;

[0029] Figure 6b is a schematic diagram of rendering a target image when there are two key reference points provided by an embodiment of the present invention;

[0030] Figure 6cis a schematic diagram of rendering a target image when there are four key reference points provided by an embodiment of the present invention;

[0031] Figure 7a is a schematic diagram of switching a rendering image in a target image provided by an embodiment of the present invention;

[0032] Figure 7b is a schematic diagram of continuous switching of a rendered image in a target image provided by an embodiment of the present invention;

[0033] Figure 8a is a user interface diagram of a video shooting interface provided by an embodiment of the present invention;

[0034] Figure 8b is a user interface diagram of a video shooting interface provided by another embodiment of the present invention;

[0035] Figure 8c is a user interface diagram of a video shooting interface provided by another embodiment of the present invention;

[0036] Fig. 9 is a schematic block diagram of an image processing device provided by an embodiment of the present invention;

[0037] Fig.10 It is a schematic block diagram of an intelligent terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to add effect materials to the target object during the process of shooting images or videos, a method of adding effect materials such as image pendants or flowing lines to the target object of the image to be processed can be adopted. In one embodiment, the image pendant refers to an image rendered on a partial image part or a specified image position of the target object during the process of shooting images or videos. The flowing line effect refers to an image that adds dynamic light and shadow flowing line effects to the edge of the target object during the process of shooting videos or images, so as to achieve the effect of lines flowing on the target object. Among them, the target object can be one or more of a person, an animal, or other objects specified by the user. In the embodiments of the present invention, the target object is mainly described as a person. When the target object is other objects, refer to the embodiments of the present invention.

[0039] In order to add the above-mentioned effect material to the target object, an embodiment of the present invention proposes an image processing method, which can improve the positioning accuracy of the effect material when it is added to the target object, so that the effect material added to the target object can better fit the target object. Specifically, the image processing method is based on the image key points of the target object. When the target object is a person, the image key points are the human body key points. In one embodiment, when determining the human body key points of the target person, the image to be processed during the shooting process can be obtained through the system application programming interface (Application Programming Interface, API) of the smart terminal. The image to be processed can be a picture image during the picture shooting process, or a video frame image during the video shooting process.

[0040] After the intelligent terminal obtains the target image to be processed, the target image (i.e., the image to be processed) can be sent to a detector for key point detection, so as to obtain a key point set for the target object (e.g., a person) in the image to be processed. In one embodiment, the target object can be, for example, a person. Then, by performing key point recognition on the person in the object to be processed, a human body key point set of the person can be obtained. Figure 1 As shown, if the target object in the image to be processed is Figure 1 In the figure, the person marked with 10, after the intelligent terminal sends the image to be processed to the detector for key point detection, the detector can perform key point recognition on the person 10, thereby obtaining a key point set of the person 10. The person after key point recognition can be Figure 1 The image is marked with 11, which is understandable. Figure 1 The image marked with 11 is only an exemplary illustration of a person image obtained after the image marked with 10 is subjected to human body key point recognition.

[0041] In one embodiment, when the smart terminal identifies key points of the target object to obtain a key point set, the identified key points may be numbered in sequence, so that the smart terminal can distinguish the multiple key points obtained according to the numbering. In one embodiment, each key point in the key point set may be numbered in sequence starting from any part of the person, for example, the key points of the person in 11 may be numbered in sequence starting from the left shoulder. After numbering each key point in the key point set, the key points included in any part of the target object may be determined, such as the key points corresponding to the left shoulder of the person in 11 include key points numbered 0 and 1.

[0042] After the smart terminal obtains the key point set of the target object in the image, it can add the preset rendering image to the image based on the obtained key points. In this way, when the image is displayed, the rendering image will be displayed at the corresponding position of the image at the same time, so that the rendering image and the target object can be well fitted, and the effect material is added. When adding the rendering image to the target object, the target sub-material associated with the target image where the target object is located can be obtained from the material data. In one embodiment, the material data is one selected by the user from multiple material data, and the material data includes multiple effect materials. Any effect material can include one or more rendering images. The rendering images used to add the target object in the current image in different effect materials constitute a target sub-material. For example, the material data A contains effect material a, effect material b and effect material c, and each effect material has rendering images numbered 1 to 6. If the rendering image numbered 1 in the effect material a, the rendering image numbered 1 in the effect material b, and the rendering image numbered 1 in the effect material c are all used to process the current image to be processed, then the rendering image numbered 1 in the effect material a, the rendering image numbered 1 in the effect material b, and the rendering image numbered 1 in the effect material c can be used as a sub-material.

[0043] After the smart terminal determines the target sub-material associated with the image, a key point subset can be determined from the key point set according to the rendering part indication information included in the target sub-material. The rendering part indication information is used to indicate the image part information corresponding to the target object in the image to be processed when the rendered image is added to the target object. For example, it can be the image part information of the left shoulder of the target object in the image, or the image part information of the head of the target object in the image, etc. The determined key point subset includes at least one key point of the target object in the rendering part.

[0044] Furthermore, the smart terminal can determine rendering parameters for the rendered image based on the image position of each key point in the key point subset in the image, so that the smart terminal can add the rendered image to the image according to the rendering parameters to obtain the target image, wherein the rendering parameters determined by the smart terminal include: one or more of a rendering position parameter, a rotation angle parameter or a scaling ratio parameter, and the smart terminal can determine a specific rendering position for adding the rendered image to the rendering part based on the rendering position parameter, thereby achieving accurate positioning of the rendered image on the target object and improving the positioning accuracy of the rendered image on the image; the smart terminal can adjust the angle of the rendered image when it is added to the target object according to the rotation angle parameter, and the smart terminal can also adjust the ratio of the rendered image when it is added to the image according to the scaling ratio parameter, so that the rendered image can be better fitted with the target object and the rendering effect is improved.

[0045] In one embodiment, see Figure 2 , is a schematic flow chart of an image processing method proposed in an embodiment of the present invention, the image processing method can be applied to the above-mentioned intelligent terminal, the intelligent terminal can be, for example, a smart phone, a tablet computer or a laptop computer, etc. Figure 2 As shown, first, the smart terminal performs key point recognition on the target object in the target image to be processed in S201 to obtain a key point set of the target object. The key point set includes multiple image key points. The target image to be processed may be an image frame in the video recording process. The smart terminal may use the current frame in the video recording process as the target image to be processed, or may use a historical image frame as the target image. In addition, the target image may also be the current image or historical image captured by the smart terminal. The target object in the target image may be a person or an animal, etc. In the embodiment of the present invention, the target object is a person as an example for detailed description. When the target object is other objects, refer to the embodiment of the present invention.

[0046] In one embodiment, after determining the target image, the smart terminal may use a key point detection algorithm to perform key point detection on the target object to obtain a set of human body key points of the person in the target image. When the key point detection algorithm is used to detect the human body to obtain a set of key points, an identification mark may be added to the obtained image key points. The identification mark may be, for example, a number mark or a unique identification code. Based on the identification mark, the smart terminal may distinguish different obtained image key points, for example, distinguish different image key points in the set of key points according to different numbers.

[0047] While the smart terminal determines the key point set of the target object from the target image, the smart terminal can obtain the target sub-material data associated with the target image from the material data in S202. Figure 3 As shown, any material data may include at least one effect material, such as material data A includes effect material a, effect material b and effect material c, and any of the effect materials corresponds to at least one rendered image, such as effect material a includes rendering images 1 to 6. The user can select any material data from the user interface of the smart terminal, and the smart terminal obtains the target sub-material data associated with the target image from the selected material data according to the user's selection.

[0048] In one embodiment, the target sub-material data is sub-material data obtained from the material data and used to process the current image to be processed (i.e., the target image), and the target sub-material data includes rendered images from different effect materials used to add to the current image to be processed, such as the target sub-material data associated with the target image includes rendered image 1 of effect material a, rendered image 1 of effect material b, and rendered image 1 of effect material c in material data A. The target sub-material data also includes rendering part indication information of the rendered image, and the rendering part indication information is used to indicate the image position where the rendered image is added to the target image, and the rendering part indication information may be, for example, the head or the left shoulder.

[0049] In one embodiment, the step S201 and the step S202 are not executed sequentially, and the step S201 may be executed first and then the step S202, or the step S202 may be executed first and then the step S201. After the smart terminal determines the target sub-material data associated with the target image, in S203, a key point subset may be determined from the key point set according to the rendering part indication information, and the image position of each image key point in the key point subset in the target image may be determined. Specifically, the smart terminal may determine the area where the rendered image is added to the target image according to the rendering part indication information. For example, if the rendering part indication information is the left shoulder, the smart terminal determines that the rendered image will be added to the left shoulder area of ​​the target object in the target image, so that the rendered image may be added according to the determined area. In this way, when the target image is displayed, the rendered image may also be displayed in the same image area of ​​the target image such as the left shoulder area.

[0050] In one embodiment, since the rendering part indication information indicates the part corresponding to the target object when performing image rendering, a key point subset can be determined from the key point set in combination with the number of each image key point obtained by the smart terminal when performing key point recognition on the target object. Among them, each image key point determined in the key point subset is obtained after the smart terminal performs numbering on the corresponding part of the target object. For example, if the rendering part indication information is the left shoulder, then according to the number of each image key point in the key point set, if the image key points numbered 0 and 1 are determined to be the image key points obtained by performing key point recognition on the left shoulder of the target object, then the key point subset determined from the key point set according to the rendering part indication information includes the image key points numbered 0 and 1.

[0051] After determining the key point subset according to the rendering part indication information, the image position of each image key point in the key point subset in the target image can be determined, so that the smart terminal can add the rendered image at the determined image position. In one embodiment, when determining the image position of each image key point in the target image, the smart terminal can determine the image position of the image key point on the target image based on the position of the image key point corresponding to the target object, wherein the smart terminal can mark the determined image position with pixel coordinates.

[0052] After the smart terminal determines the image position of each image key point in the target image, in S204, rendering parameters can be determined for the rendered image according to the image position of each image key point in the target image, and the rendered image can be added to the target image according to the rendering parameters. In this way, the rendered image can be displayed overlappingly on the basis of the target image, and the final effect after the rendering is added is as mentioned later. Figure 7a In one embodiment, the rendering parameters include a rendering position parameter and a rendering adjustment parameter, the rendering position parameter is used to determine the target image position where the rendered image is rendered onto the target image, the rendering adjustment parameter includes a rotation angle parameter or a scaling parameter, or the rendering adjustment parameter includes the rotation angle parameter and the scaling parameter, the rotation angle parameter is used by the smart terminal to adjust the angle of the rendered image when adding the rendered image, and the scaling parameter is used to adjust the size of the rendered image when adding the rendered image.

[0053] When determining the rendering parameters, the intelligent terminal may determine the rendering parameters according to the number of rendering reference points indicated in the target sub-material data. When the number of rendering reference points indicated in the target sub-material data is one, the number of key reference points corresponding to the rendering reference points in the target image is also one, so that the image position of the key reference point in the target image can be used as the rendering position parameter when the rendering image is rendered, wherein the image position can be a coordinate position in the target image, and the corresponding rendering position parameter is the coordinate value. Further, when determining the rotation angle parameter, the intelligent terminal may re-determine two image key points for calculating the rotation angle parameter from the key point subset, so that the straight line segments corresponding to the two image key points can be determined based on the two determined image key points, which are equivalent to the angle relative to the reference coordinate axis; further, the angle of the rendering image in the target sub-material data can be obtained, so that the rotation angle parameter for the rendering image when adding the rendering image can be determined based on the angle of the straight line segment relative to the reference coordinate axis and the angle.

[0054] When the smart terminal determines the scaling parameters, it is also necessary to determine two image key points for calculating the scaling parameters from the key point subset. The image key points for calculating the scaling may be the same as, or different from, or partially the same as the two image key points for calculating the rotation angle. After determining the two image key points, the length of the straight line segment connecting the two key points may be calculated. After obtaining the length of the rendered image from the target sub-material data, the scaling ratio when adding the rendered image may be determined based on the length of the straight line segment connecting the two key points and the length of the rendered image.

[0055] In one embodiment, if the number of rendering reference points indicated in the target sub-material data is N, then according to the rendering part indication information, N key reference points corresponding to the N rendering reference points can be determined from the target image, wherein N is a positive integer greater than or equal to 2. When N=2, the intelligent terminal can determine the rendering position parameters according to the image positions of the two key reference points in the target image, and the rendering position parameters include the coordinate values ​​of the two key reference points in the target image. When two key reference points are determined, the intelligent terminal can directly calculate the rotation angle parameter or the scaling parameter based on the two determined key reference points, or directly calculate the rotation angle parameter and the scaling parameter based on the two determined key reference points at the same time. In addition, the intelligent terminal can also re-determine the reference points for rotation angle calculation and scaling calculation from the key point subset. In one embodiment, when N=4, the intelligent terminal can determine four key reference points corresponding to the four rendering reference points from the target image according to the rendering part indication information, and use the area surrounded by the four key reference points in the target image as the rendering position parameter.

[0056] After the smart terminal determines the rendering parameters, the rendered image can be added to the target image based on the determined rendering position parameters. At the same time, the smart terminal can adjust the size of the rendered image added to the target image based on the scaling parameters, or can modify the angle of the rendered image added to the target image based on the rotation angle parameters, or can also adjust the size of the rendered image added to the target image based on the scaling parameters and modify the angle of the rendered image added to the target image based on the rotation angle parameters, so that the smart terminal adds the rendered image at the rendering position, and the rendered image and the target object in the target image can fit well.

[0057] In an embodiment of the present invention, after determining the target image to be processed, the intelligent terminal can perform key point recognition on the target object in the target image to obtain a key point set of the target object. After acquiring the target sub-material data associated with the target image from the material data, a key point subset can be determined from the key point set according to the rendering part indication information included in the target sub-material data, so that rendering parameters can be determined for the rendering image included in the target sub-material data according to the image position of each image key point in the key point subset in the target image, and the rendering image can be added to the target image according to the rendering parameters, so that the determined rendering position is more accurate, which is conducive to better fitting the rendering image and the target object and improving the rendering effect.

[0058] In one embodiment, see Figure 4 , is a schematic flow chart of an image processing method proposed in another embodiment of the present invention, such as Figure 4 As shown, the smart terminal may first perform key point recognition on the target object in the target image to be processed in S401 to obtain a key point set of the target object, wherein the key point set includes multiple image key points, wherein while the smart terminal determines the key point set, it may obtain target sub-material data associated with the target image from the material data in S402, wherein the target sub-material data includes: a rendered image and rendering part indication information of the rendered image, and after the smart terminal determines the target sub-material data, it may further determine a key point subset from the key point set according to the rendering part indication information in S403, and determine the image position of each image key point in the key point subset in the target image.

[0059] In one embodiment, after the smart terminal determines the image position of each image key point in the key point subset in the target image, it can determine the rendering adjustment parameters for the rendered image in S404 according to the rendering reference point on the rendered image indicated by the target sub-material data. In one embodiment, the rendering adjustment parameters include rotation angle parameters. When the smart terminal determines the rendering adjustment parameters for the rendered image according to the rendering reference point on the rendered image indicated by the target sub-material data, it can first determine at least two image key points from the key point subset. The two determined image key points can be any two key points in the key point subset, or can also be two image key points determined according to the rendering reference point.

[0060] After the smart terminal determines at least two image key points, it can determine the first angle of the at least two key points in the target image. If the smart terminal determines exactly two image key points, the line between the two determined image key points is the determined straight line segment. If the smart terminal determines three or more image key points, the smart terminal can perform curve fitting based on the three or more image key points, thereby fitting a straight line segment that is closest to each image key point, and determine a first angle based on the straight line segment, wherein the first angle is the angle between the straight line segment determined by the at least two key points and the reference coordinate axis, and the reference coordinate axis is a coordinate axis preset by the smart terminal, such as a horizontal axis or a vertical axis. Figure 5a As shown, the intelligent terminal selects two image key points from the key point subset, the selected image key points are points numbered 0 and 1 respectively, and the first angle between the line between the image key points and the horizontal axis can be marked with α.

[0061] After the smart terminal determines the first angle, the second angle of the rendered image can be obtained from the target sub-material. In one embodiment, the second angle of the rendered image is preset in the target sub-material data, such as Figure 5b As shown, the second angle of the rendered image preset in the target sub-material can be marked with β. It can be understood that the process of marking the first angle of the target image and the second angle of the rendered image will not be displayed on the user interface, but will only be executed in the background of the smart terminal to distinguish different angles. After the smart terminal determines the first angle and the second angle, the angle difference between the first angle and the second angle can be used as the rotation angle parameter, that is, the value of the rotation angle parameter is α-β, or β-α.

[0062] In one embodiment, the rendering adjustment parameters also include scaling parameters. When the intelligent terminal determines the rendering parameters for the rendered image according to the rendering reference points on the rendered image indicated by the target sub-material data, it can also first determine at least two image key points from the key point subset. The two determined image key points and the image key points determined when calculating the rotation angle can be the same or different. After determining at least two image key points, the actual distance between the at least two image key points in the target image can be determined. The actual distance refers to the length of the straight line segment determined by the at least two image key points. In one embodiment, if exactly two image key points are determined from the key point subset, the length of the line segment between the two determined image key points is the actual distance; if three or more image key points are determined from the key point subset, a straight line segment fitting can be performed based on the three or more determined image key points, and the length of the straight line segment obtained by fitting is used as the actual distance.

[0063] After the smart terminal determines the actual distance between at least two image key points in the target image, in order to determine the scaling parameter, the standard distance of the rendered image can also be obtained from the target sub-material. In one embodiment, the standard distance of the rendered image is pre-set in the target sub-material data. After determining the standard distance of the rendered image, further, the ratio between the actual distance and the standard distance can be used as the scaling parameter.

[0064] In one embodiment, the smart terminal may also determine rendering parameters for the rendered image according to the image positions of the target image at the image key points of each image. Specifically, the smart terminal may first determine four key reference points from the key point subset, and determine the image positions of the four key reference points in the target image. After determining the four key reference points, the smart terminal may use the determined image positions of the four key reference points as rendering position parameters, so that the rendered image may be rendered in the target image according to the rendering position parameters. The target sub-material data indicates four rendering reference points, and the positions of the four rendering reference points on the target image are determined according to the rendering position parameters. That is to say, four rendering reference points are pre-set in the target sub-material. The smart terminal may select image key points corresponding to the four rendering reference points from the key point subset as four key reference points according to the four rendering reference points pre-set in the target sub-material.

[0065] In one embodiment, after determining the rendering adjustment parameters for the rendered image, the intelligent terminal may, in S405, obtain the image position of the key reference point in the target image, wherein the key reference point is determined based on each key point in the key point subset, and the key reference point may be one or N, wherein N is a positive integer greater than or equal to 2. When the rendering reference point on the rendered image indicated in the target sub-material data includes one, the key reference point is a central position point determined based on the image position of each key point in the key point subset, wherein the central position point determined based on the image position of each key point in the key point subset may be an image key point in the key point subset, or may not be an image key point in the key point subset.

[0066] In one embodiment, when the rendering reference points indicated in the target sub-material data on the rendered image include N, the key reference points are N image key points selected from the key point subset. It can be understood that the N key reference points determined by the intelligent terminal are less than or equal to the number of all key points included in the key point subset, that is, the value range of N is: greater than or equal to 2, and less than or equal to the number of all key points included in the key point subset. In one embodiment, since the key point subset is determined based on the rendering part indication information, the intelligent terminal can determine the number of key points corresponding to different rendering parts by combining the numbering of each key point when identifying the key points of the target object, so as to further determine the number of all key points of the key point subset corresponding to the part, and thus determine the value range of N. For example, when the number of all key points in the key point subset is 3, the value range of N is greater than or equal to 2, and less than or equal to 3. In one embodiment, when the target sub-material data indicates two rendering reference points on the rendered image, the key reference points are two image key points selected from the key point subset; or, when the target material data indicates four rendering parameter points, the key reference points are four image key points determined from the key point subset.

[0067] After the smart terminal determines the image position of the key reference point in the target image, in S406, the image position of the key reference point in the target image can be used as a rendering position parameter, and the rendered image can be added to the target image according to the rendering position parameter and the rendering adjustment parameter. In one embodiment, if the determined key reference point is one, the key reference point is a central position point determined according to the positions of each key point in the key point subset in the target image, and the rendering position parameter determined by the smart terminal is the position parameter of the central position point. Therefore, when rendering the rendered image in the target image according to the position parameter of the central position point and the rendering adjustment parameter, Figure 6a As shown, if the target image to be processed is determined to be the image marked with 61, and the rendered image is the image marked with 62, if a key reference point determined after key point detection of the target object included in the target image, that is, the center position point determined based on the key point subset is the point marked with 610 in the target image 61, the center position point is at the midpoint of the left shoulder of the target object of the target image, and if the preset rendering reference point in the determined rendered image 62 is 620.

[0068] When the smart terminal adds the rendered image 62 to the target image 61, the rendering reference point 620 is first overlapped with the center position point 610. If it is found that the rendered image after the rendering reference point 620 is overlapped with the center position point 610 does not fit the target image well, the angle of the rendered image 62 in the target image 61 is adjusted according to the rotation angle parameter included in the rendering adjustment parameters, and the size of the rendered image 62 in the target image 61 is adjusted based on the scaling parameter included in the rendering adjustment parameters, so that the rendered image and the target image fit each other. The image obtained after the smart terminal adds the rendered image to the target image can be represented by the image marked with 63.

[0069] In one embodiment, if the number of key reference points determined by the smart terminal is two, the key reference points are two image key points determined from the key point subset, and when the smart terminal adds the rendered image to the target image according to the two determined image key points, Figure 6b As shown, two key reference points on the target image may be determined first, assuming they are 601 and 602, and two rendering reference points on the rendered image corresponding to the two key reference points 601 and 602, assuming they are 603 and 604, respectively, wherein the rendering reference point 603 corresponds to the key reference point 601, and the rendering reference point 604 corresponds to the key reference point 602.

[0070] When the intelligent terminal adds the rendered image to the target image, the two rendering reference points may be overlapped with the two key reference points according to the correspondence between the rendering reference points and the key reference points. In one embodiment, the two rendering reference points may be overlapped with the two key reference points by modifying the image position parameters corresponding to the rendering reference points to the image position parameters corresponding to the key reference points corresponding thereto, so that the rendering reference points and the key reference points overlap in the target image. Specifically, when the two rendering reference points are overlapped with the corresponding two key reference points, any rendering reference point may be overlapped with the corresponding key reference point first, such as overlapping the rendering reference point 603 with the corresponding key reference point 601. After the rendering reference point 603 and the key reference point 601 are overlapped, in order to overlap the rendering reference point 604 and the key reference point 602, the angle and size of the rendered image in the target image can be adjusted based on the rendering adjustment parameters so that the rendering reference point 604 and the key reference point 602 overlap, thereby achieving the addition of the rendered image to the target image.

[0071] In one embodiment, if there are four determined key reference points, the four key reference points are four image reference points determined from the key point subset, and when the intelligent terminal adds the rendered image to the target image according to the four key reference points, if the positions of the determined four key reference points in the target image are as follows: Figure 6c As shown, they are marked with 10, 11, 12 and 13 respectively. When the determined rendering image is added to the target image, four rendering reference points in the rendering image corresponding to the key reference points can be determined first. It is assumed that the determined four rendering reference points are points marked with 20, 21, 22 and 23 respectively, wherein rendering reference point 20 corresponds to key reference point 10, rendering reference point 21 corresponds to key reference point 11, rendering reference point 22 corresponds to key reference point 12, and rendering reference point 23 corresponds to key reference point 13.

[0072] When the intelligent terminal adds the rendered image to the target image, the region division of the image surrounded by the four rendering reference points on the rendered image can be determined according to the processing rules of the rendering processor. The region division is preset in the rendered image. The figure composed of the four rendering reference points can be shown as the gray dotted line in the figure, and the region division based on the image composed of the four rendering reference points can be shown as the black dotted line in the figure. Figure 6c As shown, the preset area division method divides the rendered image into triangular image areas, namely area 201 and area 202. The corresponding smart terminal will divide the figure surrounded by four key reference points into corresponding areas according to the preset area division of the rendered image, such as Figure 6c As shown, the figure surrounded by four key reference points is divided into a figure including areas 101 and 102, wherein area 101 corresponds to area 201 in the rendered image, and area 102 corresponds to area 202 in the rendered image. Therefore, when the smart terminal adds the rendered image to the target image, it can add the rendered image to the target image based on the correspondence between the areas.

[0073] In one embodiment, when the number of determined key reference points is four, the intelligent terminal may also directly add the rendered image to the target image based on the correspondence between the four rendering key points determined in the rendered image and the four reference key points determined in the target image.

[0074] After the intelligent terminal adds the rendered image to the target image, if the target image is an image frame in the currently recorded video, it can also detect whether the currently recorded image frame meets the rendering switching condition during the recording of the video, and the rendering switching condition refers to the switching condition of the rendered image used to render the target image. When the intelligent terminal detects that the currently recorded image frame meets the rendering switching condition, it can determine the target image to be processed based on the currently recorded image frame, and obtain the target sub-material data associated with the target image, wherein the currently recorded image frame meets the rendering switching condition means: the currently recorded image frame is a specified image frame, or the currently recorded image frame is an image frame that reaches a specified time during the recording process; or, the currently recorded image frame meets the rendering switching condition can also mean: the currently recorded image frame is a specified image frame, and the currently recorded image frame is an image frame that reaches a specified time during the recording process. That is, during the video recording process, if the smart terminal detects that the currently recorded frame is a specified image frame, wherein the specified image frame may be, for example, an image frame pre-set in the smart terminal, or an image frame reaching a specified duration, or an image frame pre-set by the smart terminal and reaching a specified duration, then the currently rendered image in the target image is switched, such as Figure 7a As shown, when the intelligent terminal detects that the rendering switching condition is met, the rendering image rendered in the target image 701 is switched to obtain image 702.

[0075] In one embodiment, if the target image is an image frame determined from the currently recorded video, the smart terminal can also detect the rhythm point of the current music during the recording of the video. If the rhythm point of the current music is detected as the target rhythm point, the target image frame to be processed is determined according to the currently recorded image frame, and the target sub-material data associated with the target image is obtained. That is, when the rhythm point of the current music is determined to be the target rhythm point, the smart terminal switches the rendering image currently added to the target image. Alternatively, during the process of recording the video, the action of the target object can be detected. If the action of the target object is detected to be a preset action, the target image frame to be processed is determined according to the currently recorded image frame, and the target sub-material data associated with the target image is obtained, wherein the preset action includes an expression action or a gesture action, or the preset action can also include the expression action and the gesture action, the preset expression action can be, for example, opening the mouth or blinking, etc., the preset gesture action can be, for example, raising hands or akimbo, etc., so the preset action can be opening the mouth or raising hands, or opening the mouth and raising hands, etc. That is to say, the smart terminal can detect whether the target object performs a preset action during the video recording process, and when the preset action of the target object is detected, the rendering image currently added to the target image is switched.

[0076] In one embodiment, Figure 7b As shown, if the rendered image is marked with 70 in the current image, if it is detected that the rendering switching condition is met, it means that the current image frame has changed, and the target sub-material data associated with the current image frame is re-determined, thereby determining the rendered image used to render the current image frame, wherein, after the rendered image is switched, the current image frame is added to obtain an image marked with 71, and so on, an image marked with 72 can also be obtained. The smart terminal can switch and add different rendered images to the target image in sequence by detecting expressions, gestures and music rhythm points, so as to achieve the flow effect of the pattern shown by the rendered image on the target image, etc., thereby improving the rendering effect of the smart terminal on the target image.

[0077] In an embodiment of the present invention, after determining the target image to be processed, the intelligent terminal may first identify the key points of the target object in the target image, so that when the key point set of the target object is obtained, the target sub-material data managed with the target image is determined, so that a key point subset can be determined from the key point set according to the rendering part indication information, and according to the image position of each key point in the key point subset in the target image, further, the rendering adjustment parameters can be determined according to the rendering reference point on the rendered image indicated by the target sub-material data, and after obtaining the image position of the key reference point in the target image, the image position is used as the rendering position parameter, so that the rendered image can be rendered in the target image according to the rendering position parameter and the rendering adjustment parameter. Since the determined rendering position parameters are related to the image key points when rendering the rendered image, when the position of the image key points in the target image changes, the rendering position of the rendered image in the target image will also change accordingly, so that the effect of the rendered image changing with the change of the target object can be achieved, so that the rendered image can be better fitted with the target image.

[0078] In one embodiment, the above image processing method can be applied to video shooting software or image shooting software. In the embodiment of the present invention, the application scenario corresponding to the above image processing method is specifically described by taking video shooting software A as an example. When a startup instruction for video shooting software A in the smart terminal is detected, the video shooting software A is started, and when a click instruction for shooting a video is detected, the current user interface can be switched to the following example: Figure 8a The video shooting interface shown in the figure includes: a video shooting area 81 and a function selection area 82, wherein the video shooting area 81 includes: a magic selection button 810, a music selection button 811, a video recording button 812, a local viewing button 813 and an interactive selection button 814. The function selection area 82 includes: a voice change button 820, a countdown button 821, a beautification button 822, a lens rotation button 823, and more buttons 824, wherein when a selection instruction for the more buttons 824 is detected, other function options will be displayed, such as a flash function option, a fixed-point stop function option, an anti-shake function option and a virtual frame function option.

[0079] When the user shoots a video or image through the video shooting interface, if a selection instruction of the magic selection button 810 is detected, the following output may be given: Figure 8bAs shown in the interface schematic diagram, in the user interface corresponding to the magic selection button 810, multiple material data are included. The user can select any material data as the material data for processing the target object in the shooting process through the user interface. The smart terminal can obtain the target sub-material data associated with the target image from the selected material data according to the material data selected by the user, and add the rendered image included in the target sub-material data to the target image. The obtained image can be Figure 7b In the image marked with 70. In addition, if the smart terminal receives a selection instruction for the music selection button 812 during the video shooting process, any music can be selected from the current music database as the background music during the video recording process.

[0080] In one embodiment, after determining the material data to be processed for the target object, the user can continue to press the video recording button 812 to put the image acquisition device in a video recording state. When it is detected that the pressing of the video recording button 812 stops, the video recording is stopped and the operation buttons for the currently recorded video are displayed, such as Figure 8c The delete button 815 and the next button 816 shown are shown, and the duration of the currently recorded video is displayed at the video recording button 812, such as 2s. When the continuous pressing operation of the video recording button 812 is detected again, the recording is continued based on the video recorded last time to obtain the target video. In the process of recording the target video, the intelligent terminal will identify the key points of the target object in the target image to be processed, and add the target image where the target object is located based on the rendered image in the selected material data. During the video recording process, the intelligent terminal can perform voice change processing on the captured sound through the voice change button 820 included in the function selection area 82 in the user interface, or it can also perform delayed shooting processing according to the countdown button 821, or it can also perform beautification processing on the target object in the video according to the beautification button 822, or it can also click the lens rotation button 823 to rotate the shooting lens during the video shooting process, so that the recorded video is more in line with the user's needs.

[0081] Based on the description of the above-mentioned image processing method embodiment, the embodiment of the present invention further provides an image processing device, which may be a computer program (including program code) running in the above-mentioned intelligent terminal. The operation control device may execute as follows: Figure 2 and Figure 4 For the image processing method described, see Fig. 9, the image processing device includes: an identification unit 901, an acquisition unit 902, a determination unit 903 and a processing unit 904. The identification unit 901 is used to perform key point identification on a target object in a target image to be processed, and obtain a key point set of the target object, wherein the key point set includes a plurality of image key points; the acquisition unit 902 is used to obtain target sub-material data associated with the target image from material data, wherein the target sub-material data includes: a rendered image and rendering part indication information of the rendered image; the determination unit 903 is used to determine a key point subset from the key point set according to the rendering part indication information, and determine the image position of each image key point in the key point subset in the target image; the processing unit 904 is used to determine rendering parameters for the rendered image according to the image position of each image key point in the target image, and add the rendered image in the target image according to the rendering parameters.

[0082] In one embodiment, when the processing unit 904 determines rendering parameters for the rendered image at the image position of the image according to the image key points, and adds the rendered image to the target image according to the rendering parameters, it is specifically used to perform: determining rendering adjustment parameters for the rendered image according to the rendering reference points on the rendered image indicated by the target sub-material data, the rendering adjustment parameters including rotation angle parameters or scaling parameters, or the rendering adjustment parameters including rotation angle parameters and scaling parameters; obtaining the image position of the key reference point in the target image, the key reference point being determined based on each key point in the key point subset; using the image position of the key reference point in the target image as a rendering position parameter, and adding the rendered image to the target image according to the rendering position parameter and the rendering adjustment parameters.

[0083] In one embodiment, if the rendering reference point on the rendered image indicated in the target sub-material data includes one, the key reference point is a central position point determined based on the image position of each key point in the key point subset; if the rendering reference points on the rendered image indicated in the target sub-material data include N, the key reference point is N image key points selected from the key point subset, wherein N is a positive integer greater than or equal to 2.

[0084] In one embodiment, the rendering adjustment parameters include rotation angle parameters. When the processing unit 904 determines the rendering adjustment parameters for the rendered image based on the rendering reference points on the rendered image indicated by the target sub-material data, it is specifically used to execute: determining at least two image key points from the key point subset; determining a first angle of the at least two image key points in the target image, wherein the first angle is the angle between a straight line segment determined by the at least two image key points and a reference coordinate axis; obtaining a second angle of the rendered image from the target sub-material; and using the angle difference between the first angle and the second angle as the rotation angle parameter.

[0085] In one embodiment, the rendering adjustment parameters include scaling parameters. When the processing unit 904 determines the rendering adjustment parameters for the rendered image based on the rendering reference point on the rendered image indicated by the target sub-material data, the processing unit 904 is specifically used to perform: determining at least two image key points from the key point subset, and determining the actual distance between the at least two image key points in the target image, wherein the actual distance refers to the length of the straight line segment determined by the at least two image key points; obtaining the standard distance of the rendered image from the target sub-material; and using the ratio between the actual distance and the standard distance as the scaling parameter.

[0086] In one embodiment, when the processing unit 904 determines rendering parameters for the rendered image according to the image positions of the target image according to the image key points, and adds the rendered image to the target image according to the rendering parameters, it is specifically used to perform: determining four key reference points from the key point subset, and determining the image positions of the four key reference points in the target image; using the image positions of the determined four key reference points as rendering position parameters, and adding the rendered image to the target image according to the rendering position parameters; wherein four rendering reference points are indicated in the target sub-material data, and the positions of the four rendering reference points on the target image are determined according to the rendering position parameters.

[0087] In one embodiment, the target image is an image frame determined from a currently recorded video, and the acquisition unit 902 is further configured to, during the recording process of the video, if it is detected that the currently recorded image frame satisfies a rendering switching condition, determine a target image to be processed according to the currently recorded image frame, and acquire target sub-material data associated with the target image; wherein the currently recorded image frame satisfies the rendering switching condition means that: the currently recorded image frame is a specified image frame, or the currently recorded image frame is an image frame that reaches a specified duration during the recording process, or,

[0088] The currently recorded image frame satisfies the rendering switching condition when: the currently recorded image frame is a designated image frame, and the currently recorded image frame is an image frame that reaches a designated duration during the recording process.

[0089] In one embodiment, the target image is an image frame determined from a currently recorded video, and the device further includes: a detection unit 905. The detection unit 905 is used to detect the rhythm point of the current music during the recording of the video; the acquisition unit 902 is further used to determine the target image frame to be processed according to the currently recorded image frame if the detection unit 905 detects that the rhythm point of the current music is the target rhythm point, and acquire the target sub-material data associated with the target image.

[0090] In one embodiment, the target image is an image frame determined from a currently recorded video, and the method further includes: the detection unit 905 is further used to detect the action of the target object during the recording of the video; the acquisition unit 902 is further used to determine the target image frame to be processed based on the currently recorded image frame if the detection unit 905 detects that the action of the target object is a preset action, and acquire target sub-material data associated with the target image, wherein the preset action includes an expression action or a gesture action, or the preset action includes the expression action and the gesture action.

[0091] In the embodiment of the present invention, after determining the target image to be processed, the recognition unit 901 may perform key point recognition on the target object in the target image to obtain a key point set of the target object. After the acquisition unit 902 acquires the target sub-material data associated with the target image from the material data, the determination unit 903 may determine a key point subset from the key point set according to the rendering part indication information included in the target sub-material data, so that the processing unit 904 may determine rendering parameters for the rendered image included in the target sub-material data according to the image position of each image key point in the key point subset in the target image, and add the rendered image to the target image according to the rendering parameters, so that the determined rendering position is more accurate, which is conducive to better fitting the rendered image and the target object and improving the rendering effect.

[0092] See also Fig.10 , is a schematic block diagram of the structure of an intelligent terminal provided by an embodiment of the present invention, such as Fig.10The intelligent terminal in the embodiment of the present invention shown may include: one or more processors 101; one or more input devices 102, one or more output devices 103 and a memory 104. The processor 101, the input device 102, the output device 103 and the memory 104 are connected via a bus 105. The memory 102 is used to store a computer program, the computer program includes program instructions, and the processor 101 is used to execute the program instructions stored in the memory 102.

[0093] The memory 104 may include a volatile memory, such as a random-access memory (RAM); the memory 104 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the memory 104 may also include a combination of the above types of memory.

[0094] The processor 101 may be a central processing unit (CPU). The processor 101 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The PLD may be a field-programmable gate array (FPGA), a generic array logic (GAL), etc. The processor 101 may also be a combination of the above structures.

[0095] In the embodiment of the present invention, the memory 104 is used to store a computer program, the computer program includes program instructions, and the processor 101 is used to execute the program instructions stored in the memory 104 to implement the above-mentioned Figure 2 and Figure 4 The steps of the corresponding method in .

[0096] In one embodiment, the processor 101 is configured to call the program instructions to execute: performing key point recognition on a target object in a target image to be processed to obtain a key point set of the target object, wherein the key point set includes multiple image key points; obtaining target sub-material data associated with the target image from material data, wherein the target sub-material data includes: a rendered image and rendering part indication information of the rendered image; determining a key point subset from the key point set according to the rendering part indication information, and determining an image position of each image key point in the key point subset in the target image; determining rendering parameters for the rendered image according to each image key point at the image position of the target image, and adding the rendered image to the target image according to the rendering parameters.

[0097] In one embodiment, when the processor 101 determines rendering parameters for the rendered image at the image position of the image according to the image key points, and adds the rendered image to the target image according to the rendering parameters, it is also configured to call the program instructions to execute: determining rendering adjustment parameters for the rendered image according to the rendering reference points on the rendered image indicated by the target sub-material data, the rendering adjustment parameters including rotation angle parameters or scaling parameters, or the rendering adjustment parameters including the rotation angle parameters and the scaling parameters; obtaining the image position of the key reference point in the target image, the key reference point being determined based on the key points in the key point subset; using the image position of the key reference point in the target image as a rendering position parameter, and adding the rendered image to the target image according to the rendering position parameter and the rendering adjustment parameters.

[0098] In one embodiment, if the rendering reference point on the rendered image indicated in the target sub-material data includes one, then the key reference point is a central position point determined based on the image position of each key point in the key point subset; if the rendering reference point on the rendered image indicated in the target sub-material data includes N, then the key reference point is N image key points selected from the key point subset, wherein N is a positive integer greater than or equal to 2.

[0099] In one embodiment, the rendering adjustment parameters include rotation angle parameters. When the processor 101 determines the rendering adjustment parameters for the rendered image based on the rendering reference points on the rendered image indicated by the target sub-material data, the processor 101 is also configured to call the program instructions to execute: determining at least two image key points from the key point subset; determining a first angle of the at least two image key points in the target image, wherein the first angle is the angle between a straight line segment determined by the at least two image key points and a reference coordinate axis; obtaining a second angle of the rendered image from the target sub-material; and using the angle difference between the first angle and the second angle as the rotation angle parameter.

[0100] In one embodiment, the rendering adjustment parameter includes a scaling parameter, and when the processor 101 determines the rendering adjustment parameter for the rendered image according to the rendering reference point on the rendered image indicated by the target sub-material data, the processor 101 is further configured to call the program instruction to execute:

[0101] At least two image key points are determined from the key point subset, and the actual distance between the at least two image key points in the target image is determined, wherein the actual distance refers to the length of the straight line segment determined by the at least two image key points; the standard distance of the rendered image is obtained from the target sub-material; and the ratio between the actual distance and the standard distance is used as the scaling parameter.

[0102] In one embodiment, when the processor 101 determines rendering parameters for the rendered image according to the image positions of the target image according to the image key points, and adds the rendered image to the target image according to the rendering parameters, it is also configured to call the program instructions to execute: determining four key reference points from the key point subset, and determining the image positions of the four key reference points in the target image; using the determined image positions of the four key reference points as rendering position parameters, and adding the rendered image to the target image according to the rendering position parameters; wherein four rendering reference points are indicated in the target sub-material data, and the positions of the four rendering reference points on the target image are determined according to the rendering position parameters.

[0103] In one embodiment, the target image is an image frame determined from a currently recorded video, and the processor 101 is configured to call the program instruction and is further configured to execute: during the recording process of the video, if it is detected that the currently recorded image frame meets the rendering switching condition, the target image to be processed is determined according to the currently recorded image frame, and the target sub-material data associated with the target image is obtained; wherein the currently recorded image frame meets the rendering switching condition means: the currently recorded image frame is a specified image frame, or the currently recorded image frame is an image frame that reaches a specified duration during the recording process, or,

[0104] The currently recorded image frame satisfies the rendering switching condition when: the currently recorded image frame is a designated image frame, and the currently recorded image frame is an image frame that reaches a designated duration during the recording process.

[0105] In one embodiment, the target image is an image frame determined from a currently recorded video, and the processor 101 is configured to call the program instructions and is also used to execute: during the recording process of the video, detecting the rhythm point of the current music; if the rhythm point of the current music is detected to be the target rhythm point, determining the target image frame to be processed based on the currently recorded image frame, and obtaining the target sub-material data associated with the target image.

[0106] In one embodiment, the target image is an image frame determined from a currently recorded video, and the processor 101 is configured to call the program instructions and is also used to execute: during the recording of the video, detecting the action of the target object; if the action of the target object is detected to be a preset action, determining the target image frame to be processed based on the currently recorded image frame, and obtaining target sub-material data associated with the target image, wherein the preset action includes an expression action or a gesture action, or the preset action includes the expression action and the gesture action.

[0107] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0108] What is disclosed above is only a partial embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims

1. An image processing method, characterized in that: include: Performing key point recognition on a target object in a target image to be processed to obtain a key point set of the target object, wherein the key point set includes a plurality of image key points; The target image includes an image frame determined from a currently recorded video. During the recording process of the video, a rhythm point of the current music is detected, and when the rhythm point of the current music is detected to be a target rhythm point, the target image is determined according to the currently recorded image frame; During the recording process of the video, when it is detected that the currently recorded image frame satisfies the rendering switching condition, a target image to be processed is determined according to the currently recorded image frame; the currently recorded image frame satisfies the rendering switching condition when: the currently recorded image frame is an image frame that reaches a specified time length during the recording process, or the currently recorded image frame satisfies the rendering switching condition when: the currently recorded image frame is a specified image frame, and the currently recorded image frame is an image frame that reaches a specified time length during the recording process; Acquire target sub-material data associated with the target image from the material data, the target sub-material data including: a rendered image and rendering location indication information of the rendered image; wherein any material data includes at least one effect material, and any effect material corresponds to at least one rendered image, and the rendered images included in the target sub-material data include rendered images in different effect materials; Determine a key point subset from the key point set according to the rendering part indication information, and determine the image position of each image key point in the key point subset in the target image; Determine rendering parameters for the rendered image at the image position of the target image according to the image key points, and add the rendered image to the target image according to the rendering parameters; there are four key reference points. When adding the determined rendered image to the target image, first determine four rendering reference points in the rendered image corresponding to the key reference points, determine the area division of the image surrounded by the four rendering reference points on the rendered image, and perform corresponding area division on the figure surrounded by the four key reference points, perform area correspondence between the area division result of the image surrounded by the four rendering reference points and the area division result of the image surrounded by the four key reference points, so as to add the rendered image to the target image according to the area correspondence; the key reference points are four image key points determined from the key point subset.

2. The method according to claim 1, characterized in that The step of determining rendering parameters for the rendered image at the image position of the image according to each image key point, and adding the rendered image to the target image according to the rendering parameters, comprises: Determining, according to a rendering reference point on the rendered image indicated by the target sub-material data, a rendering adjustment parameter for the rendered image, wherein the rendering adjustment parameter includes a rotation angle parameter or a scaling ratio parameter, or the rendering adjustment parameter includes the rotation angle parameter and the scaling ratio parameter; Acquire an image position of a key reference point in the target image, wherein the key reference point is determined according to each key point in the key point subset; The image position of the key reference point in the target image is used as a rendering position parameter, and the rendered image is added to the target image according to the rendering position parameter and the rendering adjustment parameter.

3. The method according to claim 1, characterized in that The step of determining rendering parameters for the rendered image at the image position of the target image according to each image key point, and adding the rendered image to the target image according to the rendering parameters, comprises: Determining four key reference points from the key point subset, and determining image positions of the four key reference points in the target image; Using the determined image positions of the four key reference points as rendering position parameters, and adding the rendered image to the target image according to the rendering position parameters; Wherein, four rendering reference points are indicated in the target sub-material data, and the positions of the four rendering reference points on the target image are determined according to the rendering position parameters.

4. The method according to claim 1, characterized in that The target image is an image frame determined from a currently recorded video, and the method further includes: During the recording of the video, detecting the movement of the target object; If the action of the target object is detected to be a preset action, the target image to be processed is determined according to the currently recorded image frame, and the target sub-material data associated with the target image is obtained, wherein the preset action includes an expression action or a gesture action, or the preset action includes the expression action and the gesture action.

5. An image processing device, characterized in that: include: An identification unit, configured to perform key point identification on a target object in a target image to be processed, and obtain a key point set of the target object, wherein the key point set includes a plurality of image key points; The target image includes an image frame determined from a currently recorded video. During the recording process of the video, a rhythm point of the current music is detected, and when the rhythm point of the current music is detected to be a target rhythm point, the target image is determined according to the currently recorded image frame; During the recording process of the video, when it is detected that the currently recorded image frame satisfies the rendering switching condition, a target image to be processed is determined according to the currently recorded image frame; the currently recorded image frame satisfies the rendering switching condition when: the currently recorded image frame is an image frame that reaches a specified time length during the recording process, or the currently recorded image frame satisfies the rendering switching condition when: the currently recorded image frame is a specified image frame, and the currently recorded image frame is an image frame that reaches a specified time length during the recording process; an acquisition unit, configured to acquire target sub-material data associated with the target image from the material data, wherein the target sub-material data includes: a rendered image and rendering location indication information of the rendered image; wherein any material data includes at least one effect material, and any effect material corresponds to at least one rendered image, and the rendered images included in the target sub-material data include rendered images in different effect materials; a determining unit, configured to determine a key point subset from the key point set according to the rendering part indication information, and determine an image position of each image key point in the key point subset in the target image; A processing unit is used to determine rendering parameters for the rendered image at the image position of the target image according to the image key points, and add the rendered image to the target image according to the rendering parameters; there are four key reference points. When adding the determined rendered image to the target image, four rendering reference points in the rendered image corresponding to the key reference points are first determined, and the area division of the image surrounded by the four rendering reference points on the rendered image is determined, and the figure surrounded by the four key reference points is divided into corresponding areas, and the area division result of the image surrounded by the four rendering reference points is matched with the area division result of the image surrounded by the four key reference points, so as to add the rendered image to the target image according to the corresponding relationship of the areas; the key reference points are four image key points determined from the key point subset.

6. An intelligent terminal, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, they are used to execute the image processing method according to any one of claims 1 to 4.

Images