Image processing method, device, computer equipment and storage medium

Abstract

The present invention discloses an image processing method, device, computer equipment and storage medium. The method comprises: obtaining a target image, wherein the target image includes a facial image of a target face; monitoring a target operation instruction when a user performs image editing processing on the facial image, comparing the positional relationship between the position of the action point corresponding to the target operation instruction and the position of the facial image, so as to determine whether the target operation instruction triggers a preset image editing action; when the target operation instruction triggers a preset image editing action, adjusting the position of the key facial points to be adjusted in the facial image according to the change of the action point in the target operation instruction. The present invention can freely adjust the facial image according to the user's preference, better meet the user's adjustment accuracy requirements, and improve the user experience; it is also suitable for pinching face operations of images of different face shapes, solving the problem of incompatibility with different face shapes and enhancing applicability.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, computer equipment and storage medium. Background Art

[0002] Face pinching is a behavioral expression, that is, expressing one's like or dislike for someone. In the field of image processing technology, face pinching generally refers to DIY data operations on the appearance in an image. In the prior art, after a user uses a mobile phone or other device with a camera function to collect an image to form a user's face image, the face pinching operation is achieved by adjusting the progress bar or changing the material ball map of the model. These implementation methods usually cannot achieve the face pinching effect in a precise manner. In addition, there is also the problem of incompatibility for different face shapes. Summary of the invention

[0003] The purpose of the present invention is to solve at least one of the above-mentioned technical defects, especially the technical defects of being unable to achieve a fine face pinching effect and having an incompatibility problem for different face shapes.

[0004] In order to solve the above technical problems, the present invention provides an image processing method, comprising the following steps:

[0005] Acquire a target image, wherein the target image includes a facial image of a target face;

[0006] monitoring a target operation instruction of a user when performing image editing processing on the face image, and comparing a positional relationship between a position of an action point corresponding to the target operation instruction and a position of the face image to determine whether the target operation instruction triggers a preset image editing action;

[0007] When the target operation instruction triggers a preset image editing action, the position of the facial key points to be adjusted in the facial image is adjusted according to the change of the action point in the target operation instruction.

[0008] Optionally, the step of monitoring a target operation instruction of a user when performing image editing processing on the face image, and comparing a positional relationship between a position of an action point corresponding to the target operation instruction and a position of the face image to determine whether the target operation instruction triggers a preset image editing action includes:

[0009] Acquire facial key point data of the facial image from the target image, and generate a first area covering the facial image according to the facial key point data, so that the user can perform an image editing operation on the facial image in the first area;

[0010] monitoring a target operation instruction of a user when performing image editing processing on the face image, so as to identify position data of an action point corresponding to the target operation instruction in the face image and generate a second area representing a change of the action point corresponding to the target operation instruction according to the position data;

[0011] The first area is compared with the second area in terms of positional relationship to determine whether the target operation instruction triggers a preset image editing action.

[0012] Optionally, the step of acquiring facial key point data of the facial image from the target image and generating a first area covering the facial image according to the facial key point data comprises:

[0013] Acquire facial key point data of the facial image from the target image according to a preset facial key point extraction rule;

[0014] Describing a facial contour of the facial image according to the facial key point data, and identifying a first position representing a geometric center of the facial contour;

[0015] Generating a first area covering the face contour with the first position as a symmetric center, so that the user can perform an image editing operation on the face image in the first area;

[0016] Optionally, before the step of acquiring the facial key point data of the facial image from the target image according to a preset facial key point extraction rule, the method may further include:

[0017] An image recognition model for extracting facial key points is pre-trained to generate facial key point extraction rules for obtaining facial key point data of the facial image.

[0018] Optionally, the pre-trained image recognition model may be configured with extraction rules for the facial key points when acquiring facial key point data of the facial image.

[0019] Optionally, the step of comparing the positional relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action includes:

[0020] calculating the degree of overlap between the first area and the second area;

[0021] Detecting whether the degree of overlap is greater than a preset degree of overlap threshold;

[0022] When the degree of overlap is greater than a preset degree of overlap threshold, it is determined that the target operation instruction triggers a preset image editing action.

[0023] Optionally, the step of comparing the positional relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action includes:

[0024] detecting whether the second area is within the first area;

[0025] When the second area is located within the first area, it is determined that the target operation instruction triggers a preset image editing action.

[0026] Optionally, when the target operation instruction triggers a preset image editing action, the step of adjusting the position of the facial key points to be adjusted in the facial image according to the change of the action point in the target operation instruction includes:

[0027] Identifying the operation intention corresponding to the target operation instruction;

[0028] According to the operation intention, the correspondence between the preset operation intention and the facial key point change rule is traversed to obtain the facial key point change rule corresponding to the operation intention;

[0029] The positions of the facial key points to be adjusted in the facial image are adjusted according to the acquired facial key point change rules.

[0030] In order to solve the above technical problems, the present invention further provides an image processing device, comprising:

[0031] An acquisition module, used for acquiring a target image, wherein the target image includes a face image of a target face;

[0032] a processing module, configured to monitor a target operation instruction of a user when performing image editing processing on the face image, and compare a positional relationship between a position of an action point corresponding to the target operation instruction and a position of the face image, so as to determine whether the target operation instruction triggers a preset image editing action;

[0033] The execution module is used to adjust the position of the facial key points to be adjusted in the facial image according to the change of the action point in the target operation instruction when the target operation instruction triggers a preset image editing action.

[0034] Optionally, the image processing device further includes:

[0035] A first processing submodule is used to obtain facial key point data of the facial image from the target image, and generate a first area covering the facial image according to the facial key point data, so that the user can perform image editing operations on the facial image in the first area;

[0036] A second processing submodule is used to monitor the target operation instruction of the user when performing image editing processing on the face image, so as to identify the position data of the action point corresponding to the target operation instruction in the face image and generate a second area representing the change of the action point corresponding to the target operation instruction according to the position data;

[0037] The first execution submodule is used to compare the position relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action.

[0038] Optionally, the image processing device further includes:

[0039] A first acquisition submodule, used for acquiring facial key point data of the facial image from the target image according to a preset facial key point extraction rule;

[0040] A third processing submodule, configured to depict a facial contour of the facial image according to the facial key point data, and identify a first position representing a geometric center of the facial contour;

[0041] The second execution submodule is used to generate a first area covering the face contour with the first position as the symmetry center, so that the user can perform an image editing operation on the face image in the first area.

[0042] Optionally, the image processing device further includes:

[0043] The first training submodule is used to pre-train an image recognition model for extracting facial key points to generate facial key point extraction rules for obtaining facial key point data of the facial image.

[0044] Optionally, the image processing device further includes:

[0045] The first configuration submodule is used for the pre-trained image recognition model to configure the extraction rules of the facial key points when acquiring the facial key point data of the facial image.

[0046] Optionally, the image processing device further includes:

[0047] A first calculation submodule, used for calculating the overlap between the first area and the second area;

[0048] A fourth processing submodule, used for detecting whether the overlap degree is greater than a preset overlap degree threshold;

[0049] The third execution submodule is used to determine that the target operation instruction triggers a preset image editing action when the overlap degree is greater than a preset overlap degree threshold.

[0050] Optionally, the image processing device further includes:

[0051] A fifth processing submodule, configured to detect whether the second area is located within the first area;

[0052] The fourth execution submodule is used to determine that the target operation instruction triggers a preset image editing action when the second area is located within the first area.

[0053] Optionally, the image processing device further includes:

[0054] A first identification submodule, used to identify the operation intention corresponding to the target operation instruction;

[0055] A sixth processing submodule, configured to traverse the correspondence between preset operation intentions and facial key point change rules according to the operation intention, so as to obtain the facial key point change rules corresponding to the operation intention;

[0056] The fifth execution submodule is used to adjust the positions of the facial key points to be adjusted in the facial image according to the acquired facial key point variation rules.

[0057] To solve the above technical problem, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above image processing method.

[0058] In order to solve the above technical problem, the present invention also provides a storage medium storing computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the above image processing method.

[0059] The beneficial effects of the present invention are:

[0060] The present invention obtains a target image to be processed by an image processing operation, wherein the target image includes a facial image of a target face; the terminal monitors the target operation instruction of the user when performing image editing processing on the facial image, and compares the positional relationship between the position of the action point corresponding to the target operation instruction and the position of the facial image to determine whether the target operation instruction triggers a preset image editing action. When the judgment result is that the target operation instruction triggers a preset image editing action, the position of the facial key points to be adjusted in the facial image is adjusted according to the change of the action point in the target operation instruction, thereby realizing the gesture pinching face operation on the facial image. The above method realizes that when performing image processing on an image, the user can freely adjust the facial image according to his or her own preferences, which is easier to meet the user's adjustment accuracy requirements and improves the user experience; and it can be applied to pinching faces on images of various different face shapes, solving the problem of incompatibility with different face shapes and enhancing applicability.

[0061] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 A schematic diagram of a basic method flow of an image processing method provided by an embodiment of the present invention;

[0064] Figure 2 A schematic flow chart of a method for determining whether a target operation instruction triggers a preset image editing action in an image processing method provided by an embodiment of the present invention;

[0065] Figure 3 A schematic flow chart of a method for generating a first area covering a face image in an image processing method provided in an embodiment of the present invention;

[0066] Figure 4 A schematic flow chart of a method for comparing the positional relationship between a first area and a second area in an image processing method provided in an embodiment of the present invention;

[0067] Figure 5 A schematic flow chart of a method for comparing the positional relationship between a first area and a second area in an image processing method provided in an embodiment of the present invention;

[0068] Figure 6 A flow chart of a method for adjusting the position of key facial points to be adjusted in a facial image according to the change of the action point of an operation gesture in an image processing method provided by an embodiment of the present invention;

[0069] Figure 7 A basic structural block diagram of an image processing device provided by an embodiment of the present invention;

[0070] Figure 8 A basic structural block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0072] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or in parallel, and the sequence number of the operation is only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to different types.

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0074] Example

[0075] It will be understood by those skilled in the art that the "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of performing two-way communications on a two-way communication link. Such devices may include: cellular or other communication devices, which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / intranet access, a web browser, a notepad, a calendar and / or a GPS (Global Positioning System) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with a radio frequency receiver. The "terminal" or "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or adapted and / or configured to operate locally, and / or in a distributed form, at any other location on the earth and / or in space. The "terminal" or "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, such as a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0076] The user terminal mentioned in this embodiment is the terminal mentioned above.

[0077] See also Figure 1 , Figure 1 A schematic diagram of a basic method flow of an image processing method provided by an embodiment of the present invention.

[0078] like Figure 1 As shown, the image processing method comprises the following steps:

[0079] S100: Acquire a target image, wherein the target image includes a facial image of a target face.

[0080] Face pinching generally refers to DIY data operations on the facial features in an image. The image processing method provided by the present invention can be applied to image editing and processing in a camera or character face shaping in a game. In this embodiment, when a user uses a terminal to perform functions such as the above-mentioned operations, the target image of the image editing and processing operation to be performed can be obtained by shooting with a terminal camera or calling from the terminal memory, and the target image is sent to the background server corresponding to the function, so that the background server obtains the target image. The target image includes a facial image of the target face, and the facial image is clear and discernible.

[0081] S200: monitoring a target operation instruction of a user when performing image editing processing on the face image, and comparing a positional relationship between a position of an action point corresponding to the target operation instruction and a position of the face image to determine whether the target operation instruction triggers a preset image editing action;

[0082] In this embodiment, taking the camera APP as an example, the user displays the target image in the terminal display interface by starting the image editing function of the camera APP. The user can perform gesture operations on the terminal display interface to perform image editing on the face image of the target face in the target image, wherein the gestures can be divided into single-point adjustment operation gestures of single-point touch and linkage adjustment operation gestures of double-point touch according to the operation form. The terminal display interface is provided with a monitoring event, by monitoring the target operation instruction of the user when performing image editing on the face image on the terminal display interface, when the target operation instruction of the user performing image editing on the face image on the terminal display interface is monitored, the corresponding operation gesture in the target operation instruction and the position of the action point of the operation gesture are parsed, and the position relationship between the action point position of the operation gesture and the position of the face image in the target image is compared to determine whether the target operation instruction triggers the preset image editing action. Wherein, the action point of the operation gesture is its touch point on the terminal display interface. The positional relationship between the action point of the operation gesture and the location of the facial image in the target image can be determined based on whether the action point of the operation gesture is in the location of the facial image. By determining whether the target operation instruction triggers a preset image editing action, it can be verified whether the operation gesture is the operation gesture used by the user when performing image editing processing on the facial image in the target image. When the target operation instruction triggers a preset image editing action, it means that the operation gesture is the operation gesture used by the user when performing image editing processing on the facial image in the target image. The background server can perform image editing processing on the facial image according to the preset image editing action to implement the gesture pinching operation on the facial image; otherwise, it means that the target gesture is not the operation gesture used by the user when performing image editing processing on the facial image in the target image. The background server does not need to perform image editing processing on the facial image according to the preset image editing action, thereby enhancing the accuracy of gesture pinching.

[0083] S300: When the target operation instruction triggers a preset image editing action, the positions of the facial key points to be adjusted in the facial image are adjusted according to the changes of the action points in the target operation instruction.

[0084] In this embodiment, when the positional relationship between the action point position corresponding to the target operation instruction and the position of the facial image is compared, and the comparison result is whether the target operation instruction triggers a preset image editing action, it means that the operation gesture performed by the user is used to perform a pinching operation on the facial image. At this time, the background server can adjust the position of the target facial key points to be adjusted in the target face according to the change of the action point of the operation gesture. In this embodiment, the background server side obtains the position information of the action point of the operation gesture in the target image according to the operation form of the operation gesture parsed in the target operation instruction, and obtains the facial key points to be adjusted in the facial image according to the position information. For example, if the operation gesture is a single-point adjustment operation gesture of a single-point touch type, when the user uses the single-point touch terminal display interface using the single-point touch type operation gesture, the coordinate point position of the touch point of the operation gesture corresponding to the target image is obtained, and the facial key points matching the coordinate point positions corresponding to the touch points are identified in the target image, and the identified facial key points are the facial key points to be adjusted in the facial image. For another example, if the operation gesture is a two-point touch linkage adjustment operation gesture, when the user uses the two-point touch operation gesture to touch the terminal display interface, the coordinate point positions of the two touch points of the operation gesture corresponding to the target image are obtained, and the facial key points matching the coordinate point positions corresponding to the two touch points are identified in the target image, and the identified facial key points are the facial key points to be adjusted in the facial image. It can be understood that after obtaining the facial key points to be adjusted, other facial key points associated with the facial key points to be adjusted can also be identified according to the facial key points to be adjusted, so as to connect the facial key points to be adjusted with the facial key points associated with them to form a line or a closed area, and the formed line or closed area is the contour line or closed area to be adjusted in the facial image. For example, if the two touch points of the two-point touch operation gesture are located on the nose wing, the contour line formed by connecting all the key points on the nose wing can be used as the contour line to be adjusted in the target face. For another example, if the two touch points of the target gesture are located at the two corners of the eyes, the closed area formed by connecting all the key points for adjusting the eye can be used as the closed area to be adjusted in the target face.

[0085] The image processing method described in the above embodiment obtains the target image to be processed, wherein the target image includes the face image of the target face; monitors the target operation instruction of the user when performing image editing processing on the face image through the terminal, and compares the position relationship between the action point position corresponding to the target operation instruction and the position of the face image to determine whether the target operation instruction triggers the preset image editing action. When the judgment result is that the target operation instruction triggers the preset image editing action, the position of the face key points to be adjusted in the face image is adjusted according to the change of the action point in the target operation instruction, thereby realizing the gesture pinching face operation on the face image. The above method realizes that when performing image processing on the image, the user can freely adjust the face image according to his or her own preferences, which is easier to meet the user's adjustment accuracy requirements and improve the user experience; and it can be applied to pinching face operations on images of various different face shapes, solving the problem of incompatibility with different face shapes and enhancing applicability.

[0086] In some embodiments, see Figure 2 , Figure 2 A schematic flow chart of a method for determining whether a target operation instruction triggers a preset image editing action in an image processing method provided by an embodiment of the present invention.

[0087] like Figure 2 As shown, in step S200, when determining whether the target operation instruction triggers a preset image editing action, the following steps may be included:

[0088] S210: Acquire facial key point data of the facial image from the target image, and generate a first area covering the facial image according to the facial key point data, so that the user can perform image editing operations on the facial image in the first area.

[0089] In this embodiment, after the background server obtains the target image, face recognition technology is used to perform face recognition on the target image, so as to identify the face image of the target face in the target image and obtain the face key point data corresponding to the face image, wherein the face key point data is the coordinate point data corresponding to the key points representing the facial features in the face image. After extracting the face key point data corresponding to the face image, a first area covering the face image is generated in the target image according to the face key point data, so that the user can perform a face pinching operation on the face image in the first area. Furthermore, the first area is adaptive so that the first area always covers the face image.

[0090] In some embodiments, see Figure 3 , Figure 3A schematic flow chart of a method for generating a first area covering a facial image in an image processing method provided in an embodiment of the present invention.

[0091] like Figure 3 As shown, in the step S210, when generating the first area covering the target face, it can include: S211: acquiring facial key point data of the facial image from the target image according to a preset facial key point extraction rule; S212: depicting the facial contour of the facial image according to the facial key point data, and identifying a first position representing the geometric center of the facial contour; S213: generating a first area covering the facial contour with the first position as the symmetry center, so that the user can perform image editing operations on the facial image in the first area.

[0092] In this embodiment, face recognition technology is used to perform face recognition on the target image according to a preset face key point extraction rule, so as to identify the face image of the target face in the target image and obtain the face key point data corresponding to the face image. The face key point data includes coordinate point position data representing the feature points of the face image in the target image, and the face key points are adjustable coordinate points for performing a face pinching operation on the face image. According to the face key point data, all face key points representing the face shape are connected to depict the face contour of the target face, and a first position representing the geometric center of the face contour is identified. Then, with the first position as the symmetry center and the longest distance of the face contour as the side length, a first area covering the face contour is generated, so that the user can perform a face pinching operation on the face image in the first area.

[0093] In some embodiments, when acquiring the facial key point data corresponding to the facial image, an image recognition model for extracting facial key points can be pre-trained, and then the target image can be input into the preset image recognition model, and the image recognition model can acquire the facial key point data corresponding to the features of the facial image based on the features of the facial image. Wherein, the image recognition model is a convolutional neural network model trained to a convergence state, and the convolutional neural network model is trained to extract facial key point data. The convolutional neural network model can be a CNN convolutional neural network model or a VGG convolutional neural network model. The convolutional neural network model provided in this embodiment can train the image recognition model to a convergence state through a large amount of sample data (such as different facial images), so that the image recognition model has the ability to extract facial key point data.

[0094] Furthermore, in this embodiment, when acquiring the facial key point data corresponding to the facial image, the acquisition density of the facial key points can be adjusted according to the user's choice or the system default, so that the density of the acquired facial key points can be configurable. The greater the acquisition density, the better the face pinching effect.

[0095] S220: Monitor the target operation instructions of the user when performing image editing processing on the facial image, so as to identify the position data of the action point corresponding to the target operation instruction in the facial image and generate a second area representing the change of the action point corresponding to the target operation instruction according to the position data.

[0096] Taking the camera APP as an example, the user displays the target image in the terminal display interface by starting the image editing function of the camera APP. The user can perform gesture operations on the terminal display interface to perform image editing on the face image of the target face in the target image, wherein the gestures can be divided into single-point adjustment operation gestures of single-point touch and linkage adjustment operation gestures of double-point touch according to the operation form. In this embodiment, the target operation instruction of the user when performing image editing on the face image on the terminal display interface can be monitored, and the corresponding operation gesture in the target operation instruction and the position data of the action point of the operation gesture can be obtained, and the second area representing the change of the operation gesture can be generated according to the position data. Specifically, the position data includes the coordinate point position corresponding to the action point of the operation gesture and the change direction of the coordinate point position. After the target operation instruction of the user performing image editing on the face image on the terminal display interface is monitored, the coordinate point position corresponding to the action point of the operation gesture and the change direction of the coordinate point position are obtained according to the target operation instruction, and then the second area representing the change of the operation gesture can be generated according to the coordinate point position corresponding to the action point and the change direction of the coordinate point position.

[0097] For example, when the operation gesture is a single-point adjustment operation gesture of a single-point touch type, if the only action point of the single-point touch operation gesture is located at the outer corner of one eye, the outer corner of the eye is taken as the starting point, and the coordinate point position after the change of the only action point is taken as the end point, the starting point and the end point are connected to form a trajectory representing the change of the single-point touch operation gesture, and a second area covering the trajectory representing the change of the single-point touch operation gesture is generated with a fixed width, and the size of the second area changes with the change of the single-point touch operation gesture trajectory. For another example, when the operation gesture is a linkage adjustment operation gesture of a double-point touch type, if the two action points of the double-point touch operation gesture are located at the two corners of the eye, the second area covering the eye is generated with the line between the two action points as the side length, and the size of the second area changes with the change of the coordinate point position corresponding to the two action points. When the direction of the change of the coordinate point position corresponding to the two action points is to approach each other, the second area is reduced; when the direction of the change of the coordinate position corresponding to the two action points is to move away from each other, the second area is expanded.

[0098] S230: Compare the positional relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action.

[0099] In this embodiment, after obtaining the first area covering the target face according to the above step S210 and obtaining the second area representing the change of the target gesture according to the above step S220, by comparing the positional relationship between the first area and the second area, it is determined whether the change of the action point of the operation gesture in the second area is used to adjust the face image in the first area. The positional relationship between the first area and the second area can be determined based on whether the second area is within the first area. If the second area is within the first area, it means that the target operation instruction has triggered a preset image editing action; or the overlap between the first area and the second area is calculated, and the overlap between the calculated first area and the second area is compared to determine whether it is greater than a preset overlap threshold. If so, it means that the target operation instruction has triggered a preset image editing action.

[0100] In some embodiments, see Figure 4 , Figure 4 A schematic flow chart of a method for comparing the positional relationship between a first area and a second area in an image processing method provided in an embodiment of the present invention.

[0101] like Figure 4As shown, when comparing the positional relationship between the first area and the second area, the step S230 may include steps S231 to S233. Among them, S231: calculating the overlap between the first area and the second area; S232: detecting whether the overlap is greater than a preset overlap threshold; S233: when the overlap is greater than the preset overlap threshold, determining that the target operation instruction triggers a preset image editing action.

[0102] In this embodiment, after obtaining the first area covering the target face in step S210 and obtaining the second area representing the target gesture change in step S220, the total area of ​​the second area and the area covered by the second area on the first area are obtained, and the ratio of the area covered by the second area on the first area to the total area of ​​the second area is calculated as the overlap between the first area and the second area. A overlap threshold is pre-set in the background server to determine whether the target operation instruction triggers a preset image editing action. After calculating the overlap between the first area and the second area, the overlap is compared with the preset overlap threshold to detect whether the overlap is greater than the preset overlap threshold. When the overlap is greater than the preset overlap threshold, it is determined that the target operation instruction triggers the preset image editing action.

[0103] In some embodiments, see Figure 5 , Figure 5 A schematic flow chart of a method for comparing the positional relationship between a first area and a second area in an image processing method provided in an embodiment of the present invention.

[0104] like Figure 5 As shown, in the step S230, when comparing the positional relationship between the first area and the second area, it can also include: S234: detecting whether the second area is located in the first area; S235: when the second area is located in the first area, determining that the target operation instruction triggers a preset image editing action.

[0105] In this embodiment, after obtaining the first area covering the target face in step S210 and obtaining the second area representing the target gesture change in step S220, it is determined whether the area covered by the second area on the first area is equal to the total area of ​​the second area or the edge contour line corresponding to the second area and the edge contour line corresponding to the first area are identified, and it is determined whether the edge contour line corresponding to the second area is surrounded by the edge contour line corresponding to the second area, so as to detect whether the first area is within the first area. If so, it is determined that the target operation instruction triggers a preset image editing action.

[0106] In some embodiments, see Figure 6 , Figure 6 A flow chart of a method for adjusting the positions of key facial points to be adjusted in a facial image according to changes in the action points of an operation gesture in an image processing method provided in an embodiment of the present invention.

[0107] like Figure 6 As shown, the step S300 includes steps S310 to S330. Among them, S310: identifying the operation intention corresponding to the target operation instruction; S320: traversing the correspondence between the preset operation intention and the face key point change rule according to the operation intention to obtain the face key point change rule corresponding to the operation intention; S330: adjusting the position of the face key point to be adjusted in the face image according to the obtained face key point change rule.

[0108] In this embodiment, the backend server is pre-set with a corresponding relationship between the operation intention and the face key point change rule. For example, for a single-point adjustment operation gesture of a single-point touch type, the corresponding face key point change rule is that the face key point matching the only action point of the single-point adjustment operation gesture moves along the change direction of the only action point. For a double-point touch linkage adjustment operation gesture, the corresponding face key point change rule is that the two face key points matching the two action points of the linkage adjustment operation gesture and all face key points associated with the two matching face key points move along the change direction of the two action points. Among them, on the face image, the association relationship between each face key point is set according to the facial features and contour relationship of the face image. For example, on the nose wing, the contour line formed by the association of all face key points on the same side, if the two action points of the linkage adjustment operation gesture are respectively located on both sides of the nose wing and both correspond to and match any face key point on both sides of the nose wing, then all face key points on both sides of the nose wing move along the change direction of the two action points. When the change directions of the two action points are relatively inward, all the facial key points on both sides of the nose move inward. It can be understood that in this embodiment, when performing the double-touch linkage adjustment operation gesture, the distance moved by each facial key point can be equal or unequal. When the distance moved by each facial key point is unequal, the moving distance corresponding to each facial key point can be calculated by an algorithm.

[0109] In this embodiment, the operation intention corresponding to the target operation instruction is first identified, for example, whether the operation gesture in the target operation instruction is a single-point touch single-point adjustment operation gesture or a double-point touch linkage adjustment operation gesture, and the facial key points corresponding to the position of the action point of the operation gesture are identified, so as to identify the facial key points to be adjusted in the facial image. After the operation intention corresponding to the target operation instruction is identified, the correspondence between the operation intention and the facial key point change rule pre-set on the background server side is traversed according to the operation intention corresponding to the target operation instruction, so as to obtain the facial key point change rule corresponding to the operation intention, so as to adjust the position of the facial key points to be adjusted in the facial image according to the obtained facial key point change rule.

[0110] In order to solve the above technical problems, the present invention also provides an image processing device. Figure 7 , Figure 7 This is a basic structural block diagram of an image processing device provided by an embodiment of the present invention.

[0111] like Figure 7 As shown, an image processing device includes: an acquisition module, a processing module and an execution module. The acquisition module is used to acquire a target image, wherein the target image includes a facial image of a target face; the processing module is used to monitor the target operation instruction of the user when performing image editing processing on the facial image, and compare the position relationship between the action point position corresponding to the target operation instruction and the position of the facial image to determine whether the target operation instruction triggers a preset image editing action; the execution module is used to adjust the position of the facial key points to be adjusted in the facial image according to the change of the action point in the target operation instruction when the preset image editing action is triggered by the target operation instruction.

[0112] The image processing device described in the above embodiment obtains the target image to be executed for the image processing operation, wherein the target image includes the face image of the target face; monitors the target operation instruction of the user when performing image editing processing on the face image through the terminal, and compares the position relationship between the action point position corresponding to the target operation instruction and the position of the face image to determine whether the target operation instruction triggers the preset image editing action. When the judgment result is that the target operation instruction triggers the preset image editing action, the position of the face key points to be adjusted in the face image is adjusted according to the change of the action point in the target operation instruction, thereby realizing the gesture pinching face operation on the face image. The above method realizes that when performing image processing on the image, the user can freely adjust the face image according to his or her own preferences, which is easier to meet the user's adjustment accuracy requirements and improve the user experience; and it can be applied to pinching face operations on images of various different face shapes, solving the problem of incompatibility with different face shapes and enhancing applicability.

[0113] In some embodiments, the image processing device further includes: a first processing submodule, a second processing submodule and a first execution submodule. The first processing submodule is used to obtain the facial key point data of the facial image from the target image, and generate a first area covering the facial image according to the facial key point data, so that the user can perform image editing operations on the facial image in the first area; the second processing submodule is used to monitor the target operation instruction of the user when performing image editing processing on the facial image, so as to identify the position data of the action point corresponding to the target operation instruction in the facial image and generate a second area representing the change of the action point corresponding to the target operation instruction according to the position data; the first execution submodule is used to compare the position relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action.

[0114] In some embodiments, the image processing device further includes: a first acquisition submodule, a third processing submodule, and a second execution submodule. The first acquisition submodule is used to acquire the facial key point data of the facial image from the target image according to the preset facial key point extraction rule; the third processing submodule is used to depict the facial contour of the facial image according to the facial key point data, and identify the first position representing the geometric center of the facial contour; the second execution submodule is used to generate a first area covering the facial contour with the first position as the symmetry center, so that the user can perform image editing operations on the facial image in the first area.

[0115] In some embodiments, the image processing device further includes: a first training submodule. The first training submodule is used to pre-train an image recognition model for extracting facial key points to generate facial key point extraction rules for obtaining facial key point data of the target face.

[0116] In some embodiments, the image processing device further includes a first configuration submodule. The first configuration submodule is used for the pre-trained image recognition model to configure the extraction rules of the facial key points when acquiring the facial key point data of the target face.

[0117] In some embodiments, the image processing device further includes a first calculation submodule, a fourth processing submodule and a third execution submodule. The first calculation submodule is used to calculate the overlap between the first area and the second area; the fourth processing submodule is used to detect whether the overlap is greater than a preset overlap threshold; and the third execution submodule is used to determine that the target operation instruction triggers a preset image editing action when the overlap is greater than the preset overlap threshold.

[0118] In some embodiments, the image processing device further includes: a fifth processing submodule and a fourth execution submodule. The fifth processing submodule is used to detect whether the second area is located in the first area; the fourth execution submodule is used to determine that the target operation instruction triggers a preset image editing action when the second area is located in the first area.

[0119] In some embodiments, the image processing device further includes: a first recognition submodule, a sixth processing submodule and a fifth execution submodule. The first recognition submodule is used to identify the operation intention corresponding to the target operation instruction; the sixth processing submodule is used to traverse the correspondence between the preset operation intention and the face key point change rule according to the operation intention to obtain the face key point change rule corresponding to the operation intention; the fifth execution submodule is used to adjust the position of the face key points to be adjusted in the face image according to the obtained face key point change rule.

[0120] To solve the above technical problems, the present invention also provides a computer device. Figure 8 , Figure 8 A basic structural block diagram of a computer device provided in an embodiment of the present invention.

[0121] like Figure 8 As shown in Figure 1, the internal structure diagram of a computer device. Figure 8As shown, the computer device includes a processor, a non-volatile storage medium, a memory and a network interface connected via a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database and computer-readable instructions, and the database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement an image processing method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute an image processing method. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] In this embodiment, the processor is used to execute Figure 7 The specific functions of the acquisition module 10, the processing module 20 and the execution module 30 are stored in the memory, and the program code and various data required to execute the above modules are stored. The network interface is used to transmit data between the user terminal or the server. The memory in this embodiment stores the program code and data required to execute all sub-modules in the image processing device, and the server can call the program code and data of the server to execute the functions of all sub-modules.

[0123] The computer device described in the above embodiment obtains the target image to be executed for the image processing operation, wherein the target image includes the face image of the target face; monitors the target operation instruction of the user when performing image editing processing on the face image through the terminal, and compares the position relationship between the action point position corresponding to the target operation instruction and the position of the face image to determine whether the target operation instruction triggers the preset image editing action. When the judgment result is that the target operation instruction triggers the preset image editing action, the position of the face key points to be adjusted in the face image is adjusted according to the change of the action point in the target operation instruction, thereby realizing the gesture pinching face operation on the face image. The above method realizes that when performing image processing on the image, the user can freely adjust the face image according to his or her own preferences, which is easier to meet the user's adjustment accuracy requirements and improve the user experience; and it can be applied to pinching face operations on images of various different face shapes, solving the problem of incompatibility with different face shapes and enhancing applicability.

[0124] The present invention also provides a storage medium storing computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image processing method described in any of the above embodiments.

[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0126] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0127] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: The following steps are involved: Acquire a target image, wherein the target image includes a facial image of a target face; monitoring a target operation instruction of a user when performing image editing processing on the face image, and comparing a positional relationship between a position of an action point corresponding to the target operation instruction and a position of the face image to determine whether the target operation instruction triggers a preset image editing action; When the target operation instruction triggers a preset image editing action, the operation form of the operation gesture is obtained according to the analysis of the target operation instruction. If the operation gesture is a two-point touch linkage adjustment operation gesture, the coordinate point positions corresponding to the two touch points of the operation gesture in the target image are obtained, and facial key points matching the coordinate point positions corresponding to the two touch points are identified in the target image, and the positions of the facial key points are adjusted.

2. The image processing method according to claim 1, characterized in that: The step of monitoring a target operation instruction of a user when performing image editing processing on the face image, and comparing the positional relationship between the action point position corresponding to the target operation instruction and the position of the face image to determine whether the target operation instruction triggers a preset image editing action includes: Acquire facial key point data of the facial image from the target image, and generate a first area covering the facial image according to the facial key point data, so that the user can perform an image editing operation on the facial image in the first area; monitoring a target operation instruction of a user when performing image editing processing on the face image, so as to identify position data of an action point corresponding to the target operation instruction in the face image and generate a second area representing a change of the action point corresponding to the target operation instruction according to the position data; The first area is compared with the second area in positional relationship to determine whether the target operation instruction triggers a preset image editing action.

3. The image processing method according to claim 2, characterized in that: The step of acquiring facial key point data of the facial image from the target image and generating a first area covering the facial image according to the facial key point data comprises: Acquire facial key point data of the facial image from the target image according to a preset facial key point extraction rule; Describing a facial contour of the facial image according to the facial key point data, and identifying a first position representing a geometric center of the facial contour; A first area covering the face contour is generated with the first position as the symmetric center, so that the user can perform an image editing operation on the face image in the first area.

4. The image processing method according to claim 3, characterized in that: Before the step of acquiring the facial key point data of the facial image from the target image according to the preset facial key point extraction rule, the method may further include: An image recognition model for extracting facial key points is pre-trained to generate facial key point extraction rules for obtaining facial key point data of the facial image.

5. The image processing method according to claim 4, characterized in that: The pre-trained image recognition model can be configured with extraction rules for the facial key points when acquiring the facial key point data of the facial image.

6. The image processing method according to claim 2, characterized in that: The step of comparing the positional relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action includes: calculating the degree of overlap between the first area and the second area; Detecting whether the degree of overlap is greater than a preset degree of overlap threshold; When the degree of overlap is greater than a preset degree of overlap threshold, it is determined that the target operation instruction triggers a preset image editing action.

7. The image processing method according to claim 2, characterized in that: The step of comparing the positional relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action includes: detecting whether the second area is within the first area; When the second area is located within the first area, it is determined that the target operation instruction triggers a preset image editing action.

8. The image processing method according to any one of claims 6 or 7, characterized in that: The step of adjusting the positions of the key points of the face comprises: Identifying the operation intention corresponding to the target operation instruction; According to the operation intention, the correspondence between the preset operation intention and the facial key point change rule is traversed to obtain the facial key point change rule corresponding to the operation intention; The positions of the facial key points to be adjusted in the facial image are adjusted according to the acquired facial key point change rules.

9. An image processing device, characterized in that: include: An acquisition module, used for acquiring a target image, wherein the target image includes a face image of a target face; a processing module, configured to monitor a target operation instruction of a user when performing image editing processing on the face image, and compare a positional relationship between a position of an action point corresponding to the target operation instruction and a position of the face image, so as to determine whether the target operation instruction triggers a preset image editing action; The execution module is used to, when the target operation instruction triggers a preset image editing action, parse the target operation instruction to obtain the operation form of the operation gesture; if the operation gesture is a two-point touch linkage adjustment operation gesture, obtain the coordinate point positions of the two touch points of the operation gesture in the target image, and identify the facial key points in the target image that match the coordinate point positions corresponding to the two touch points, and adjust the positions of the facial key points.

10. The image processing device according to claim 9, characterized in that: Also includes: A first processing submodule is used to obtain facial key point data of the facial image from the target image, and generate a first area covering the facial image according to the facial key point data, so that the user can perform image editing operations on the facial image in the first area; A second processing submodule is used to monitor the target operation instruction of the user when performing image editing processing on the face image, so as to identify the position data of the action point corresponding to the target operation instruction in the face image and generate a second area representing the change of the action point corresponding to the target operation instruction according to the position data; The first execution submodule is used to compare the position relationship between the first area and the second area to determine whether the target operation instruction triggers a preset image editing action.

11. The image processing device according to claim 10, characterized in that: Also includes: A first acquisition submodule, used for acquiring facial key point data of the facial image from the target image according to a preset facial key point extraction rule; A third processing submodule, configured to depict a facial contour of the facial image according to the facial key point data, and identify a first position representing a geometric center of the facial contour; The second execution submodule is used to generate a first area covering the face contour with the first position as the symmetry center, so that the user can perform an image editing operation on the face image in the first area.

12. The image processing device according to claim 11, characterized in that: It may also include: The first training submodule is used to pre-train an image recognition model for extracting facial key points to generate facial key point extraction rules for obtaining facial key point data of the facial image.

13. The image processing device according to claim 12, characterized in that: It includes a first configuration submodule, which is used for the pre-trained image recognition model to configure the extraction rules of the facial key points when acquiring the facial key point data of the facial image.

14. The image processing device according to claim 10, characterized in that: Also includes: A first calculation submodule, used for calculating the overlap between the first area and the second area; A fourth processing submodule, used for detecting whether the overlap degree is greater than a preset overlap degree threshold; The third execution submodule is used to determine that the target operation instruction triggers a preset image editing action when the overlap degree is greater than a preset overlap degree threshold.

15. The image processing device according to claim 10, characterized in that: Also includes: A fifth processing submodule, configured to detect whether the second area is located within the first area; The fourth execution submodule is used to determine that the target operation instruction triggers a preset image editing action when the second area is located within the first area.

16. The image processing device according to any one of claims 14 or 15, characterized in that: Also includes: A first identification submodule, used to identify the operation intention corresponding to the target operation instruction; A sixth processing submodule, configured to traverse the correspondence between preset operation intentions and facial key point change rules according to the operation intention, so as to obtain the facial key point change rules corresponding to the operation intention; The fifth execution submodule is used to adjust the positions of the facial key points to be adjusted in the facial image according to the acquired facial key point change rules.

17. An electronic device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the image processing method according to any one of claims 1 to 8.

18. A storage medium storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the image processing method according to any one of claims 1 to 8.

Images