Image processing method and device, electronic equipment and computer readable storage medium

Abstract

An image processing method and device, electronic equipment and computer readable storage medium. The image processing method comprises: in response to an operation on a first target object in a first interface, displaying a target window in the first interface; and in response to an operation on the target window, displaying an image determined based on the operation on the target window on the first target object. The method and device provided by the embodiments of the present disclosure can display the first interface when the user needs to process the image displayed on the target object to adjust the virtual article worn by the target object, so that the user can conveniently adjust the displayed image, and at least partially reduce the design cost of the special effect of displaying the image on the target object.

Description

Technical Field

[0001] This disclosure relates to an image processing method, an image processing apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of image processing technology, applications that allow users to instantly experience clothing, makeup, and accessories through the camera lens have become widespread. For example, users can instantly experience the wearing effect of virtual items, such as how clothing looks, the application effect of different lipstick shades, and the decorative effect of different nail art patterns. The realism of these effects and the diversity of virtual items are important factors affecting the user experience. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] At least one embodiment of this disclosure provides an image processing method, including: displaying a target window in a first interface in response to an operation on a first target object in a first interface; and displaying an image on the first target object based on the operation on the target window in response to an operation on the target window.

[0005] At least another embodiment of this disclosure provides an image processing apparatus, including: a first display module configured to display a target window in a first interface in response to an operation on a first target object in a first interface; and a second display module configured to display an image on the first target object based on the operation on the target window in response to an operation on the target window.

[0006] At least one further embodiment of this disclosure provides an electronic device, including: a processing device; and a storage device including one or more computer program instructions; wherein the one or more computer program instructions are executed by the processing device to perform the image processing method provided in at least one embodiment of this disclosure.

[0007] At least one further embodiment of this disclosure provides a computer-readable storage medium that non-transitory stores computer-readable instructions, wherein the computer-readable instructions, when executed by a processor, implement the image processing method provided in at least one embodiment of this disclosure.

[0008] At least one further embodiment of this disclosure provides a computer program product, including a computer program / instructions that, when run on a computer, cause the computer to perform the image processing method provided in at least one embodiment of this disclosure. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0010] Figure 1 This illustration schematically depicts an application scenario of the image processing method and apparatus provided in at least one embodiment of the present disclosure;

[0011] Figure 2 A flowchart illustrating at least one embodiment of the image processing method of this disclosure is shown schematically.

[0012] Figure 3 This schematically illustrates a change in the first interface in response to an operation on a first target object, according to at least one embodiment of the present disclosure.

[0013] Figure 4 The illustration schematically shows a first target object displaying an image in response to an operation on a target window, according to at least one embodiment of the present disclosure.

[0014] Figure 5 The illustration schematically shows the principle that the pattern included in the image displayed on the first target object of at least one embodiment of the present disclosure changes with the adjustment length indicated by the first length adjustment control;

[0015] Figure 6 This schematically illustrates a change in the first interface in response to an operation on a target window, according to at least one embodiment of the present disclosure.

[0016] Figure 7 This schematic diagram illustrates a variation of the first interface according to at least one embodiment of the present disclosure;

[0017] Figure 8 A schematic diagram of a second interface according to at least one embodiment of the present disclosure is shown;

[0018] Figure 9 This schematic diagram illustrates the changes in the second interface in response to operation according to at least one embodiment of the present disclosure;

[0019] Figure 10 The illustration schematically shows a principle diagram of displaying an image determined based on an operation on a target window on a first target object according to at least one embodiment of the present disclosure;

[0020] Figure 11 This illustration schematically shows a principle diagram for determining target key points for each object according to at least one embodiment of the present disclosure;

[0021] Figure 12 This illustration schematically shows a principle diagram of determining an initial region for each object according to at least one embodiment of the present disclosure;

[0022] Figure 13 The illustration schematically shows a principle diagram of displaying an image determined based on an operation on a target window on a first target object according to at least another embodiment of the present disclosure;

[0023] Figure 14 A schematic block diagram of an image processing apparatus according to at least one embodiment of the present disclosure is shown; and

[0024] Figure 15 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0031] With the development of image processing technology, applications that allow users to instantly experience clothing, makeup, and accessories through the camera lens have become widespread. For example, users can instantly experience the wearing effect of virtual items, such as how clothing looks, the application effect of different lipstick shades, and the decorative effect of different nail art patterns. The realism of these effects and the diversity of virtual items are important factors affecting the user experience.

[0032] For example, making virtual objects move precisely as they move in front of the camera is one of the main ways to improve the realism of the effect, while reducing the design cost of virtual objects is one of the main ways to increase their diversity. Virtual objects can be provided, for example, as special effects assets.

[0033] For example, the wearing effect of virtual items can be achieved by generating a mask for the target region (the region where the virtual item needs to be worn) in an image based on an image segmentation algorithm. Then, the target region is fused with a base image (e.g., an image of the virtual item) based on this mask to display the virtual item in that target region. In this method, the wearing effect of the virtual item is affected by the accuracy of the image segmentation algorithm, and the virtual item cannot move with the target region in the image, resulting in poor realism. Furthermore, due to the limitations of the image segmentation algorithm's accuracy, this method usually requires the designer of the special effects materials to adjust the mask, making the design process for the special effects materials quite complex.

[0034] For example, the wearing effect of virtual items can also be achieved by first constructing a transparent model containing a 3D target object (such as the object to which the virtual item needs to be worn). The virtual item's effect is then displayed only in the target area of ​​this transparent model (part or all of the target object). Subsequently, 3D effects are used to control the transparent model, allowing the virtual item's effect to change as the target area moves. However, this method does not accurately identify and locate the target object's partial or complete area, and it involves the construction of a 3D model. This requires designers of special effects materials to bind the target object's structure to the model, making it difficult to implement and hindering widespread application.

[0035] To at least partially solve these technical problems, at least one embodiment of this disclosure provides an image processing method, the method comprising: displaying a target window in a first interface in response to an operation on a first target object in a first interface; and displaying an image on the first target object based on the operation on the target window in response to an operation on the target window.

[0036] Based on the image processing method provided in at least one embodiment of this disclosure, at least one embodiment of this disclosure also provides an image processing apparatus, an electronic device, and a computer-readable storage medium.

[0037] The image processing method provided in at least one embodiment of this disclosure can display a target window when a user needs to process the image displayed on a target object to adjust the virtual items worn by the target object. This allows the user to easily adjust the displayed image and at least partially reduces the design cost of special effects for displaying images on the target object.

[0038] The embodiments and some examples of this disclosure will now be described in detail with reference to the accompanying drawings.

[0039] Figure 1 The illustration shows an application scenario of the image processing method and apparatus provided in at least one embodiment of the present disclosure.

[0040] like Figure 1 As shown, the application scenario 100 of this embodiment includes a first user 101, a first terminal device 102, and a server 103. The first user 101 can design special effects materials through the first terminal device 102, and the first terminal device 102 can send the designed special effects materials to the server 103, which is connected to it via a network.

[0041] The first terminal device 102 can be, for example, an electronic device such as a mobile phone, tablet computer, laptop computer, or desktop computer that has the first client application installed. The first client application may include, for example, an image processing application and / or a special effects material design application, etc., and this disclosure embodiment does not limit this. For example, the first user 101 can upload images when using the first client application and complete the design of special effects materials by interacting with the image processing window (hereinafter also referred to as the "target window") displayed on the first terminal device 102. For example, the special effects materials can be materials used to provide the wearing effect of virtual items, and virtual items can be, for example, clothing, headwear, nail art, lip color, glasses, etc., and this disclosure embodiment does not limit this.

[0042] For example, such as Figure 1As shown, application scenario 100 may further include a second user 104 and a second terminal device 105. The second terminal device 105 may be, for example, an electronic device such as a mobile phone, tablet, laptop, or desktop computer with a second client application installed. The second client application may be, for example, a shopping application and / or a video recording application. This second client application may, for example, have the following functions: providing special effects materials to the user, and presenting the user with an image of wearing virtual items in response to the user's selection of special effects materials. For example, server 103 may be a backend management server supporting the operation of the second client application, and the special effects materials may be obtained by the second terminal device 105 from server 103 via a network.

[0043] For example, the first terminal device 102 and the second terminal device 105 can be the same type of electronic device, or they can be different types of electronic devices. The server supporting the first client application and the server supporting the second client application can be the same server, or they can be different servers. Figure 1 The server 103 shown can indicate a server cluster, in which at least some servers support the operation of a first client application and at least another part of servers support the operation of a second client application. The servers in the server cluster can communicate with each other to transmit at least some data. For example, at least some servers can transmit special effects materials designed by the first client application in response to user operations to at least another part of the servers.

[0044] For example, the image processing method provided in at least one embodiment of this disclosure can be implemented in software, hardware, firmware, or any combination thereof. For example, the image processing method provided in at least one embodiment of this disclosure is applicable to a first terminal device 102 or a second terminal device 105, which can load and execute the image processing method; the embodiments of this disclosure do not limit this. For example, the first terminal device 102 and / or the second terminal device 105 may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or other processing units with data processing capabilities and / or instruction execution capabilities, storage units, etc. The computing device may also be equipped with an operating system, application programming interfaces (e.g., OpenGL (Open Graphics Library), Metal, etc.), etc., and implement the image processing method provided in the embodiments of this disclosure by running code or instructions. For example, the computing device may also include an output component, such as a display component, which may be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a quantum dot light-emitting diode (QLED) display, etc., and the embodiments disclosed herein are not limited thereto. For example, the display component may display user interaction pages such as special effects material design pages and / or special effects material usage pages.

[0045] Figure 2 A flowchart illustrating at least one embodiment of the image processing method of this disclosure is shown. Figure 3 The illustration schematically shows a change in the first interface in response to an operation on a first target object, according to at least one embodiment of the present disclosure.

[0046] like Figure 2 As shown, the image processing method 200 of this embodiment includes steps S210 to S220.

[0047] Step S210: In response to an operation on a first target object in the first interface, a target window is displayed in the first interface.

[0048] Step S220: In response to an operation on the target window, display an image on the first target object determined based on the operation on the target window.

[0049] For example, the first interface can be a page presented by the first client application described above. The first interface may display an image, which could be a single captured image, any frame from a recorded video, or an image captured by the lens of the terminal device displaying the first interface, i.e., an image within the lens's field of view. This image includes a first target object. The first target object can be, for example, an object that needs to be tried on with a virtual item, such as any part of the body of any subject, including the head, lips, hands, feet, fingernails, toenails, etc. For example, if the first target object is a fingernail, it can include the fingernails of one or more fingers on the hand; the first target object can be the middle fingernail, or it can be the middle and index fingernail, etc. If the first target object is a toenail, it can include the toenails of one or more toes on the foot; for example, the first target object can be the toenail of the first toe, or it can be the toenail of the first toe and the toenail of the second toe, etc.

[0050] For example, an operation on the first target object could be a processing operation on an image that can be displayed on the first target object. For example, the first interface may display an entry link or entry control, and an operation on the first target object could include clicking on that entry link or entry control.

[0051] For example, the first interface may display an add control (which can be understood as an entry control) for adding an image to the first target object, and the target window may be displayed in response to the click operation of the add control.

[0052] For example, such as Figure 3 As shown, in at least one embodiment, the first interface 300 may display an add control 310 for adding effects to an object including a first target object (the display name of the control may be "Add Object"). In response to a click operation on the add control 310, multiple effect types may be displayed in the first interface, and these multiple effect types may include target types for adding effects to the first target object. In response to a selection operation on the target type, a target window may be displayed. Accordingly, operations on the first target object include clicking the add control and selecting the target type. The multiple effect types may include, for example, "body outline," "limb tracking," and "nail tracking," etc. In the case where the first target object is a nail, the target type is "nail tracking." Correspondingly, in response to operations on the target window, the image displayed on the nail is adjusted, and the pattern included in the image may be overlaid on the nail. It is understood that the above-described effect types and first target objects are merely examples to facilitate understanding of this disclosure, and the embodiments of this disclosure do not limit them.

[0053] For example, such as Figure 3As shown, the target window 330 may include an input area, and operations on the input area can be used as operations on the target window (hereinafter also referred to as "target operations"). In response to the target operation, a terminal device with a first client application installed can, for example, acquire information entered in the input area through the target operation, and determine the image based on the target operation by analyzing that information. For example, the information entered in the input area may include at least one type of information such as text, image, and voice. Display parameters can be obtained by performing text recognition, image recognition, and / or voice recognition on the input information, and the image matching the display parameters can be used as the image determined based on the target operation.

[0054] For example, display parameters may include pattern type, display length of the image including the pattern, pattern transparency, pattern color, and name of the first target object corresponding to the pattern. These display parameters can be preset as needed, and this embodiment does not limit them. For example, if the display parameters determined based on the target operation include a pattern transparency of 70%, a pattern color of pink, and a pattern type of heart, then the image displayed on the first target object may include a pink heart-shaped pattern with 70% transparency.

[0055] The effect of displaying an image on the first target object can be achieved in the following way: first, obtain the mesh model of the first target object, and then attach the image corresponding to the display parameters to the mesh model, thereby obtaining the effect of displaying the image corresponding to the display parameters on the first target object. The image includes a pattern that matches the display parameters. For example, only a part of the image is displayed on the first target object, and the area occupied by the displayed part of the image in the whole image is the area where the pattern that matches the display parameters is located.

[0056] For example, the mesh model can be a geometric structure composed of vertices, edges, and faces. For instance, this embodiment can first obtain multiple key points of the first target object, then use these key points as vertices of the geometric structure, the lines connecting two key points as edges of the geometric structure, and the regions enclosed by at least three edges as faces. This mesh model can be used to represent the surface shape of the first target object. It is understood that this mesh model can be a two-dimensional mesh model or a three-dimensional mesh model; this disclosure does not limit this.

[0057] For example, a 2D multi-view diffusion model can be used to process the image including the first target object to generate intermediate multi-view images. Then, a large reconstruction model (e.g., LRM, Large Reconstruction Model For Single Image to 3D) can be used to construct a mesh model of the first target object based on the image including the first target object and the intermediate multi-view images. Alternatively, 3D mesh generation techniques such as Fancy123 (One Image to High-Quality 3D Mesh Generation via Plug-and-Play Deformation) can be used to process the image including the first target object to obtain a mesh model of the first target object.

[0058] For example, the principle of texture mapping can be used to attach an image corresponding to the display parameters to the mesh model. For instance, the image corresponding to the display parameters can have a size set according to actual needs, and the image includes multiple pattern points, the number of which can be equal to the number of vertices in the mesh model of the first target object. The mapping relationship between these multiple pattern points and the vertices in the mesh model can be preset. Based on this mapping relationship, the image is attached to the mesh model. Thus, the texture of the surface of the first target object corresponding to the mesh model is the pattern included in the image corresponding to the display parameters. It is understood that the mapping relationship can be set according to actual needs, and this disclosure does not limit this.

[0059] For example, when the mesh model is a 3D model, UV mapping can be used to attach the image to the mesh model. UV mapping is the process of mapping the surface of a 3D model to a 2D texture coordinate system. UV is short for texture coordinates. Texture coordinates define the position information of each point on the 2D texture image. These points correspond to the vertices in the 3D model to determine the position of each point on the 2D texture image on the 3D model surface when the 2D texture image is attached to the 3D model surface.

[0060] This embodiment of the disclosure displays a target window in response to an operation on a first target object, providing a window for the user to input display parameters. This allows the user to adjust the display parameters of the image simply by operating the target window, thereby achieving the design of the image displayed on the first target object. Compared to existing technologies that require binding the structure of the target object to a model, and those that require adjusting the mask, this approach can at least partially reduce the design cost of special effects displayed on the target object, thereby increasing the design cost of special effects materials and thus improving the diversity of special effects materials (such as wearable virtual items) and enhancing the user experience.

[0061] In at least one embodiment of this disclosure, the first target object may be, for example, a fingernail, an ear, teeth, or toenails. The first target object may include at least two objects belonging to at least two types. The special effects materials for the first target object may differ for different types of objects among the at least two objects. For example, taking a fingernail as the first target object, the first target object may include the following objects: thumb, index finger, middle finger, ring finger, and little finger. Taking an ear as the first target object, the first target object may include the left ear and the right ear. Unless otherwise specified, the following description uses a fingernail as an example to describe the implementation principles of various embodiments of this disclosure; however, it is understood that the embodiments of this disclosure are not limited in this respect, and the implementation principles of various embodiments of this disclosure are similar for different first target objects.

[0062] For example, the target window to be displayed can be a window corresponding to an object of at least one of two types. Figure 3 As shown, the target window can be displayed in the following manner: In response to an operation on a first target object, at least two first identification information corresponding to at least two types are first displayed in the first interface 300; subsequently, in response to a selection operation on the first information among the at least two first identification information, a target window for a first type is displayed in the first interface, where the first type is the type corresponding to the selected first information. For example, the at least two first identification information are displayed in the selection window 320 corresponding to the first target object. For example, the at least two first identification information include the following information: thumb, index finger, middle finger, ring finger, and little finger. In response to a selection operation on the identification information "middle finger," a target window 330 corresponding to the middle finger can be displayed in the first interface 300.

[0063] For example, in response to an operation on target window 330, an image determined based on the operation on target window 330 can be displayed on a first object belonging to a first type among at least two objects included in the first target object. Figure 3 As shown in the figure below, in response to an operation on the target window 330, an image determined based on the operation on the target window 330 is displayed on the nail of the middle finger selected out of the five nails shown.

[0064] For example, the principle of displaying an image on a first object can be as follows: first, a mesh model of the first object is obtained; then, an image determined based on operations targeting a target window is pasted onto the surface of the mesh model, thereby achieving the effect of displaying the image determined based on operations targeting the target window on the first object. The principle of obtaining the mesh model of the first object is similar to the principle of obtaining the mesh model of the first target object described above, and the principle of pasting the image onto the surface of the mesh model can be the texture mapping principle described above, which will not be repeated in this embodiment. For example, the image determined based on operations targeting the target window includes a pattern that matches the display parameters determined based on operations targeting the target window.

[0065] This disclosure embodiment, by setting a target window for each type of object, facilitates the design of different patterns for different types of objects, which helps to improve the diversity of special effects materials (such as instantly wearable virtual items) designed for the first target object, and improves the user experience for both users and designers of special effects materials.

[0066] In at least one embodiment of this disclosure, in response to an operation on a first target object, the first interface may display a predetermined image on the first target object in addition to displaying a target window. Subsequently, in response to an operation on the target window, the image displayed on the first target object is adjusted in real time so that the adjusted image is an image determined based on the operation on the target window, i.e., the pattern included in the adjusted image matches the display parameters determined based on the operation on the target window. The predetermined image may be, for example, a default image set by the user in the settings page of the first client application, or an image set during the development of the first client application; this disclosure does not limit this.

[0067] For example, the aforementioned step of displaying an image on a first target object based on an operation on a target window may include: in response to an operation on the target window, adjusting a predetermined image displayed on the first target object to an image determined based on the operation on the target window. For example, if the display parameters determined based on the operation on the target window include a heart-shaped pattern, and the predetermined image includes a square pattern, then the square pattern displayed on the first target object will be replaced with a heart-shaped pattern. If the display parameters include a blue pattern color, and the predetermined image includes a pink heart-shaped pattern, then the pink heart-shaped pattern displayed on the first target object will be adjusted to a blue heart-shaped pattern. For example, the adjustment of the predetermined image displayed on the first target object can be real-time. If the operation on the target window is a series of N continuously executed operations, then the predetermined image displayed on the first target object can be adjusted N times, and each adjusted image includes a pattern matching the display parameters determined based on the real-time operation on the target window. N is a natural number greater than 1.

[0068] Understandably, when the target window is a window of the first type, in response to the operation on the target window, the predetermined image displayed on the first object of the first type will be adjusted, while the predetermined images displayed on other objects in the first target object besides the first object may not be adjusted.

[0069] This embodiment of the disclosure displays a predetermined image on the first target object in response to an operation on the first target object, and adjusts the displayed image in response to an operation on the target window. This achieves the purpose of displaying the adjustment effect of the pattern to the designer of the special effects material in real time, thereby providing guidance for the adjustment of the pattern to a certain extent, which helps to improve the design efficiency of special effects materials and improve the user experience.

[0070] In at least one embodiment of this disclosure, the target window may include a target control, and operations on the target control may be regarded as operations on the target window. The image to be displayed on the first target object may be determined based on the operations on the target control. For example, the display parameters of the image to be displayed on the first target object may be determined first based on the operations on the target control, and then the image to be displayed on the first target object may be determined based on the display parameters.

[0071] Figure 4 The illustration schematically shows a first target object displaying an image in response to an operation on a target window, according to at least one embodiment of the present disclosure.

[0072] like Figure 4 As shown, in at least one embodiment of this disclosure, the target window 430 displayed in the first interface 400 may include a first length adjustment control 431. This first length adjustment control 431 may include a slider, for example, in response to a sliding operation on the slider, the length of the image displayed on the first target object is adjusted. Accordingly, operations on the target window include sliding operations on the slider. It is understood that... Figure 4 The slider shown is merely an example to aid in understanding this disclosure. The first length adjustment control 431 may also include, for example, a text box or a dropdown list. For instance, the display length of the pattern included in the image may be determined in response to information entered in the text box. Alternatively, the display length of the pattern included in the image may be determined based on the adjustment length corresponding to any of a predefined plurality of length adjustment options provided in the dropdown list.

[0073] For example, the information indicated by the first length adjustment control is the adjusted length, which can be understood as a reference length. The base length corresponding to this reference length can be, for example, the length of the first target object. In other words, the reference length can be understood as a multiple of the base length. This embodiment can determine the display length of the image region including the pattern in the image based on the product of the adjusted length and the base length, that is, the value of the display length included in the display parameters. The length of the first target object can be understood as the length of the first target object displayed in the first interface, or it can be understood as the average length of the first target object obtained by statistics. This embodiment does not limit this.

[0074] For example, when a predetermined image is displayed on a first target object in response to an operation on a first target object, the step of displaying an image on the first target object based on an operation on a target window may include: in response to an operation on a first length adjustment control, the length of the pattern included in the predetermined image displayed on the first target object in the growth direction of the first target object changes with the adjustment length indicated by the first length adjustment control.

[0075] For example, if target window 430 is a target window for the middle finger, i.e., the window displayed in response to the selection operation of the first identification information "middle finger" in selection window 420, then in response to the sliding operation of the slider included in the first length adjustment control 431, the length of the pattern included in the image displayed on the middle fingernail of the first target object in the growth direction of the middle fingernail will change in real time. For example, if the sliding operation is to slide the slider along the extension direction of the slider, then the adjustment length indicated by the first length adjustment control increases, and the length of the pattern included in the image displayed on the middle fingernail will also increase accordingly. For example, selection window 420 is a window displayed in response to the click operation of "add object" control 410.

[0076] For example, if target window 430 is for all objects included in the first target object, then Figure 4 The lengths of the patterns included in the images displayed on the nails of the five fingers, along the nail growth direction, change with the adjustment length indicated by the first length adjustment control. For example, when determining the display length based on the adjustment length indicated by the first length adjustment control, the adjusted length of the image area including the pattern displayed on each fingernail can be determined based on the adjustment length and the reference length of each fingernail. By adjusting the length of the image area including the pattern, the length of the pattern can be adjusted.

[0077] This embodiment of the disclosure provides a target control in the target window, which facilitates the designer of special effects materials to adjust the display parameters of the image displayed on the first target object. Compared with technical solutions that require user input of display parameters, this method lowers the design threshold for special effects materials because the designer does not need to know in advance what display parameters need to be provided. Furthermore, by making the length of the pattern included in the image displayed on the first target object change with the adjustment length indicated by the first length adjustment control, the designer can easily view the effect of the display parameter adjustment in real time and determine when to complete the parameter adjustment. This eliminates the need for the designer to know the design rules of the display parameters in advance, further lowering the design threshold. In addition, by setting the first length adjustment control, the designer can easily adjust the length of the pattern displayed on the target object without having to change the image, thus improving the efficiency of pattern length adjustment.

[0078] The following will combine Figure 5 The principle that the pattern in the image displayed on the first target object changes with the adjusted length is further described and expanded.

[0079] Figure 5 The illustration schematically shows a pattern included in an image displayed on a first target object according to at least one embodiment of the present disclosure, which changes with the adjustment length indicated by a first length adjustment control.

[0080] In at least one embodiment of this disclosure, the first target object may include at least one object, which includes at least the aforementioned first object corresponding to the first type. In response to an operation of the first length adjustment control, the length of the pattern included in the predetermined image displayed on all objects included in the first target object or on the aforementioned first object corresponding to the first type changes with the adjusted length.

[0081] like Figure 5 As shown, in one embodiment 500, in response to an operation on the first length adjustment control 531, the real-time adjustment length 501 indicated by the first length adjustment control 531 can be determined first. For each object in which the length of the pattern included in the displayed predetermined image changes with the adjustment length, the boundary points included in the mesh model 502 for each object can be adjusted based on the real-time adjustment length to obtain an adjusted mesh model 503. After obtaining the adjusted mesh model 503, a predetermined image 504 can be attached to the surface of the adjusted mesh model 503 based on a predetermined mapping relationship, and the display effect of the predetermined image displayed on each object is as follows. Figure 5 The object 505 is shown in the table.

[0082] For example, the boundary points included in the mesh model 502 may include key points of each detected object, or points after expanding or adjusting the key points of each object. Taking a fingernail as an example, the position of only the top point in the mesh model 502 can be adjusted to achieve the effect of adjusting the length of the pattern included in the predetermined image in the growth direction of each object. It is understood that since the coordinate points mapped to the boundary points of the mesh model on the predetermined image in the predetermined mapping relationship are fixed, by adjusting the position of the boundary points of the mesh model, the length of the pattern included in the image displayed on the object corresponding to the mesh model can be adjusted after the predetermined image is attached to the surface of the adjusted mesh model. For example, the predetermined mapping relationship can be a UV mapping relationship, which is not limited in this embodiment.

[0083] For example, if the adjustment length indicated by the first length adjustment control increases, the coordinate values ​​of the boundary points in the growth direction of each object can be increased, and the amount of increase can be based on the length of each object in the growth direction. For example, if the adjustment length changes from 1 to 1.2, the increase in the coordinate values ​​of the boundary points in the growth direction of each object can be: (1.2-1)*base length. If the adjustment length indicated by the first length adjustment control decreases, the coordinate values ​​of the boundary points in the growth direction of each object can be decreased, and the amount of decrease can be based on the length of each object in the growth direction. For example, if the adjustment length changes from 1 to 0.8, the increase in the coordinate values ​​of the boundary points in the growth direction of each object can be: (1-0.8)*base length. It is understood that the above adjustment principles are only examples to understand this disclosure, and the embodiments of this disclosure are not limited thereto.

[0084] In at least one embodiment of this disclosure, the boundary points included in the mesh model can form multiple boundary point groups, each boundary point group including root points and top points with corresponding relationships. For example, a root point can correspond to one or at least two top points, or a top point can correspond to one or at least two root points. The correspondence between root points and top points can be set according to actual needs, and this disclosure does not limit this.

[0085] For example, such as Figure 5 As shown, in embodiment 500, the mesh model 502 may include 8 boundary points, of which 5 are top points and 3 are root points. For example, for ease of description, ... Figure 5 The five top points are numbered from left to right as: 1, 2, 3, 4, 5. Figure 5The three root points are numbered 8, 7, and 6 from left to right. The boundary point group consisting of these eight boundary points can, for example, include five boundary point groups represented by the following numbering groups: (1, 8), (2, 8), (3, 7), (4, 6), and (5, 6). For example, the numbering group represents the boundary point group consisting of the top point numbered 1 and the root point numbered 8.

[0086] For example, different strategies can be set for adjusting the boundary points for two cases: when the real-time adjustment length indicated by the first length adjustment control is greater than 1 and when the real-time adjustment length is less than 1, so that the adjusted effect of the pattern displayed on each object is more realistic.

[0087] For example, when the real-time adjustment length is less than 1, interpolation can be performed on the positions of the two endpoints in each boundary point group, and the interpolated positions can be used as the adjusted positions of the top points in each boundary point group. For instance, if the positions of the boundary points numbered 1 to 8 are p1 to p8 respectively, and the real-time adjustment length is length, then the adjusted positions p1' to p5' of the five top points numbered 1 to 5 can be calculated using the following formula. For example, the positions of the boundary points numbered 1 to 8 could be the positions when the adjustment length is 1.

[0088] p1' = p1 * length + p8 * (1 - length)

[0089] p2' = p2 * length + p8 * (1 - length)

[0090] p3' = p3 * length + p7 * (1 - length)

[0091] p4' = p4 * length + p6 * (1 - length)

[0092] p5' = p5 * length + p6 * (1 - length)

[0093] In at least one embodiment, a vector pointing from the center point of a plurality of top points to the center point of a plurality of root points can be used as an offset reference vector to adjust the position of the boundary points of the mesh model, thereby determining the adjustment direction and unit length of the adjustment of the boundary point position. For example, the offset reference vector can be a vector x pointing from the top point of the target group (3, 7) in the five boundary point groups to the root point of the target group.

[0094] For example, when the real-time adjustment length is greater than 1, the boundary point positions can be adjusted with reference to the offset reference vector. For instance, the adjusted positions of the top points included in each boundary point group can be determined based on the real-time adjustment length (length) and the offset reference vector. For example, the dot product of (length-1) and vector x can be used as the offset required for the top points included in each boundary point group. For example, this offset of the top points can be the offset relative to the position of the top points when the adjustment length is 1.

[0095] For example, when determining the offset of the top point included in each boundary point group, the adjustment ratio for each boundary point group can be determined based on the first distance between the root point and the top point included in each boundary point group and the second distance between the root point and the top point included in the target group. This adjustment ratio is the proportion by which the top point of each boundary point group is adjusted, or the adjustment coefficient for the offset of the top point can be determined. For example, the ratio of the first distance to the second distance can be used as the adjustment ratio. After determining the adjustment ratio and the offset reference vector, the adjusted position of the top point included in each boundary point group can be determined, for example, based on the adjustment ratio for each boundary point group, the real-time adjustment length, and the offset reference vector x. For example, the offset of the top point included in the five boundary point groups (1, 8), (2, 8), (3, 7), (4, 6), and (5, 6) can be expressed as follows. The offsets of the top points numbered 1 to 5 can be represented as offset1 to offset5 respectively. The distances between the top points and the root points included in each boundary point group can be represented as dis(1,8), dis(2,8), dis(3,7), dis(4,6), and dis(5,6).

[0096] offset1=dis(1,8) / dis(3,7)*(length-1)x

[0097] offset2=dis(2,8) / dis(3,7)*(length-1)x

[0098] offset3=dis(3,7) / dis(3,7)*(length-1)x

[0099] offset4=dis(4,6) / dis(3,7)*(length-1)x

[0100] offset5=dis(5,6) / dis(3,7)*(length-1)x

[0101] In determining the adjusted position of the top points included in each boundary point group, this embodiment of the present disclosure considers the adjustment ratio determined based on the first distance and the second distance, which makes the adjusted position of each top point more realistic, making the resulting adjusted mesh model more realistic, thereby making the adjusted image displayed on each object more beautiful and improving the design efficiency of special effects materials.

[0102] Figure 6 The illustration schematically shows a change in the first interface in response to an operation on a target window, according to at least one embodiment of the present disclosure.

[0103] like Figure 6 As shown, in at least one embodiment of this disclosure, the target window 630 displayed in the first interface 600 may include an image selection control 632. Depending on actual needs, the target window 630 may also include a first length adjustment control 631. The implementation principle of the first length adjustment control 631 is similar to that of the first length adjustment control 531 described above, and will not be repeated here.

[0104] For example, the image selection control 632 may include a drop-down list that provides a plurality of pre-defined image options. Any one of these image options can be used as the aforementioned pre-defined image. Figure 6 As shown, multiple alternative images can be displayed in response to an operation on the image selection control 632. For example, the multiple alternative images may include stickers... Figure 1 ,stick Figure 2 ,stick Figure 3 ,stick Figure 4 wait.

[0105] like Figure 6 As shown, in one embodiment, in response to an operation on a first target object, a predetermined image can be displayed on the first target object. After displaying a plurality of alternative images, in response to a selection operation of a target image among the plurality of alternative images, the predetermined image displayed on the first target object can be replaced with the target image. For example, in response to a sticker... Figure 4 The selection operation can replace the predetermined image displayed on the first target object with a sticker. Figure 4 That is, the pattern displayed on the first target object is replaced by a pattern included in the predetermined image. Figure 4 Includes patterns.

[0106] For example, the first target object may include at least two objects of at least two different types. This embodiment may display a predetermined image on both of the at least two objects in response to an operation targeting the first target object. When the target window is a window targeting all objects, in response to a selection operation of the target image from a plurality of candidate images, the predetermined images displayed on both of the at least two objects may be replaced with the target image. When the target window is a window targeting only a first type, in response to a selection operation of the target image from a plurality of candidate images, only the predetermined image displayed on the first object of the first type targeted by the target window may be replaced with the target image, thereby facilitating different designs for different types of objects.

[0107] For example, when the target window is a window that is only for the first type, this embodiment can also respond to an operation for the first target object by displaying a predetermined image only on the first object of the first type that the target window is targeting, and in response to a selection operation of the target image among a plurality of alternative images, replace the predetermined image displayed on the first object of the first type that the target window is targeting with the target image, so as to facilitate the design of special effects materials only for some types of objects.

[0108] For example, the pattern selection control 632 may also include a file picker, whose display name could be, for example, "Import Material". For instance, in response to an operation on this file picker, multiple files locally stored on the terminal device running the first client application can be displayed, these files may include multiple candidate images. Alternatively, in response to a further selection of a storage address, multiple files containing multiple candidate images stored on another device communicatively connected to the terminal device can be displayed. In response to a selection operation on a file containing a target image from among the multiple candidate images, the predetermined image displayed on at least a portion of the object is replaced with the target image.

[0109] For example, such as Figure 6 As shown, the image selection control 632 can include both a drop-down list and a text selector. The multiple image options provided by the drop-down list can be pre-defined during the development phase of the first client application. The text selector provides an entry point for designers of special effects materials to import their own designed images. This enriches the images available during the design phase of special effects materials, facilitating diverse design possibilities.

[0110] For example, the display name of the image selection control 632 may be "texture" or something similar, and this embodiment of the present disclosure does not limit this.

[0111] Figure 7 A schematic diagram illustrating variations of the first interface according to at least one embodiment of the present disclosure is shown.

[0112] In at least one embodiment of this disclosure, when the first target object includes at least two objects of at least two types, if an image is displayed on a second object of the second type corresponding to the second information, after a predetermined image or an image determined based on an operation on the target window is displayed on at least some of the objects included in the first target object, for example, in response to a deletion operation of the second information among the at least two first identification information displayed in the aforementioned first interface 300, the pattern displayed on the second object can be hidden. For example, the first interface may also display a deletion control corresponding to each first identification information, and the deletion operation of the second information may include a click operation on the deletion control corresponding to the second information. Alternatively, the deletion operation of the second information may include a click operation on the deletion control included in the window displayed by right-clicking the second information. It is understood that the above deletion operations are only examples to facilitate understanding of this disclosure, and the embodiments of this disclosure do not limit this.

[0113] For example, such as Figure 7 As shown, in a scenario where the first target object is a fingernail, in response to the deletion operation of the second information "middle finger" among multiple first identification information such as "thumb", "index finger", and "middle finger", the image displayed on the fingernail of the middle finger included in the first target object in the first interface 700 is hidden.

[0114] By setting up deletion operations for the first identifier information, designers of special effects materials can flexibly adjust the types of objects that need to be designed during the design process of special effects materials.

[0115] In at least one embodiment of this disclosure, the target control included in the target window 730 may, for example, include an object type selection control 733. The object type selection control 733 may, for example, include a drop-down list that provides at least two type options. Figure 7 As shown, in a scenario where the first target object is a fingernail, at least two type options can include the following: thumb, index finger, middle finger, ring finger, and little finger. For example, in response to an operation on the object type selection control, at least two second identification information corresponding to each of the at least two types can be displayed. This second identification information can be, for example, a type name, etc., and this embodiment of the disclosure does not limit this.

[0116] For example, in one embodiment, the target control may further include an image selection control 732 and a first length adjustment control 731. The implementation principles of the image selection control 732 and the first length adjustment control 731 can be found above and will not be repeated here.

[0117] For example, in response to an operation on any of the information in the second identification information, based on the operation on the target window, an image determined by the operation on the target window can be displayed on an object of the type corresponding to any of the information, that is, an image including a pattern that matches the display parameters determined by the operation on the target window.

[0118] For example, if the target window is a window of type 1 out of at least two types, then in response to a selection operation of the third information corresponding to the second type among the at least two second identification information displayed, an image with the same display parameters as the image displayed on the first object of type 1 can be displayed on the second object belonging to type 2, that is, a pattern with the same display parameters as the pattern displayed on the first object can be displayed on the second object.

[0119] For example, the second type could be the type corresponding to the second information targeted by the aforementioned deletion operation. Thus, even if a mistaken deletion operation is performed on the second information of at least two first identifiers in the first interface, the design of special effects materials for the second object belonging to the second type can be achieved by manipulating the object type selection control in the target window, thereby improving the design experience.

[0120] For example, in a scenario where the first target object is a fingernail, the display name of the object type selection control could be, for example, "Which finger?". After a deletion operation on the identifier "middle finger" from at least two first identifiers, the image displayed on the middle fingernail, which is included in the first target object, is hidden. Subsequently, in response to an operation on the object type selection control in the target window corresponding to the index finger type, at least two second identifiers, namely the identifiers "thumb," "index finger," and "middle finger," are displayed in the target window. In response to a selection operation on the third identifier "middle finger," which corresponds to the middle finger type from at least two second identifiers, an image is displayed on the middle fingernail again; that is, the image displayed on the middle fingernail is no longer hidden, and the image is adjusted to have the same display parameters as the image displayed on the index fingernail. It is understood that the display name, first identifier, and second identifier of the above-described object type selection control are merely examples to facilitate understanding of this disclosure, and the embodiments of this disclosure do not limit this.

[0121] For example, even if no deletion operation is performed on the first identification information corresponding to any type of object, an image with the same display parameters as the image displayed on the first object can be displayed on the second object in response to a selection operation of the third information among at least two second identification information displayed in the target window of the first type.

[0122] In at least one embodiment of this disclosure, the design of the image displayed on the first target object can be determined in response to a determination operation on the image displayed on the first target object. For example, in response to the determination operation, special effects material for the first target object can be automatically generated based on the image displayed on the first target object. The special effects material includes the image displayed on the first target object. For example, the first client application can pre-maintain a template for generating special effects material, and the first client application can generate special effects material by substituting the display parameters of the determined image displayed on the first target object into the template. For example, the special effects material can be implemented by a program module, and the template for generating special effects material can be a program template; this disclosure does not limit this aspect.

[0123] For example, the first interface may also display a confirmation control corresponding to the first target object. Clicking this confirmation control can be considered a confirmation operation for the image displayed on the first target object. Alternatively, clicking a confirmation key (e.g., the Enter key) on an input device (which can be a virtual keyboard depending on actual needs) can be considered a confirmation operation for the image displayed on the first target object. It is understood that the above confirmation operations are merely examples to aid understanding of this disclosure, and the embodiments of this disclosure do not limit the scope of the disclosure.

[0124] For example, after obtaining the special effects material, the first client application can send the material to the server supporting its operation, and then, through a communication connection between the servers, send the material to the server supporting the second client application. Alternatively, the server supporting the second client application can retrieve the material from the server supporting the first client application via a communication connection.

[0125] Figure 8 A schematic diagram of a second interface according to at least one embodiment of the present disclosure is shown.

[0126] In at least one embodiment of this disclosure, a terminal device with a first client application installed can simultaneously install a second client application, which can be a video shooting application, a shopping application, etc. The second client application can, for example, call the camera integrated in the terminal device and display the image within the lens's field of view in real time on the interface. For example, the second client application can also respond to user operations by displaying multiple alternative materials, which may include special effects materials generated by the aforementioned first client application. Figure 8As shown, when a second target object is displayed in the second interface 800 and a selection operation on the special effects material 810 is detected, the second client application can, for example, run a program module for implementing the special effects material 810 to display the image included in the special effects material 810 on the second target object displayed in the second interface 800. For example, the principle of displaying the image included in the special effects material on the second target object can be similar to the principle of displaying the image determined based on the operation on the target window on the first target object. The specific principle can be found below and will not be detailed here.

[0127] For example, the second target object and the aforementioned first target object can be the same subject or the same part of different subjects. For example, if the first target object is the fingernail of subject A, the second target object can be the fingernail of subject B. If the first target object is the lips of subject A, the second target object can be the lips of subject B.

[0128] In this embodiment of the disclosure, by responding to the selection operation of special effects material, the image including the special effects material is displayed on the second target object displayed in the second interface 800, which can achieve the effect of the second target object trying on virtual items.

[0129] Figure 9 The illustration shows a change in response to operating the second interface, according to at least one embodiment of the present disclosure.

[0130] In at least one embodiment of this disclosure, the template used when generating special effects material may, for example, also have the function of generating length adjustment controls. For example, such as... Figure 9 As shown, in response to the selection operation of special effects material, the second client application, by running the program module for generating special effects material based on the template, can not only display the image included in the special effects material 910 on the second target object in the second interface 900, but also display the second length adjustment control 920 in the second interface 900. The implementation principle of the second length adjustment control 920 is similar to that of the aforementioned first length adjustment control. For example, the difference between the second length adjustment control 920 and the first length adjustment control may include: in response to the operation of the second length adjustment control, for all objects displaying images, the length of the pattern included in the displayed image will change with the adjustment length indicated by the second length adjustment control; while in response to the operation of the first length adjustment control, only the length of the pattern included in the image displayed on the first type of object corresponding to the target window displaying the first length adjustment control will change with the adjustment length indicated by the first length adjustment control.

[0131] like Figure 9 As shown, the second length adjustment control may include a slider, which can be slid along the extension direction of the slider in response to a sliding operation on the slider (e.g., in...). Figure 9The image shown on the page (where the slider is slid to the right) shows patterns that are elongated in the direction of nail growth on the images displayed on the five objects included in the second target object (e.g., the five fingernails of the hand).

[0132] This embodiment of the disclosure allows users of special effects materials to conveniently adjust the length of virtual items they are trying on according to their actual needs by displaying and setting the second length adjustment control. This provides users of special effects materials with richer effects and improves their user experience.

[0133] The following will combine Figures 10-13 A detailed description of the principles behind displaying images on objects is provided.

[0134] Figure 10 The illustration schematically shows a principle diagram of displaying an image determined based on an operation on a target window on a first target object according to at least one embodiment of the present disclosure.

[0135] In at least one embodiment of this disclosure, the first target object may include at least one object. Upon detecting that the first target object is displayed in a first interface, a first client application may, for each of the at least one object, obtain a mesh model for that object based on target key points for that object. For example, the first client application may also obtain the mesh model for each object in response to an operation on the first target object. Embodiments of this disclosure do not limit the timing of obtaining the mesh model; for example, the timing of obtaining the mesh model should be earlier than the timing of displaying an image on the object.

[0136] For example, the first client application could also, upon detecting a selection operation on the first information among at least two first identifiers, obtain the mesh model of each object within the first target object that belongs to the first type corresponding to the first information. For example, as... Figure 10 As shown, the mesh model obtained for each object can be model 1001. For example, the basic structure of the obtained mesh models is similar for different types of objects, but the boundary points of the mesh models for different types of objects may differ in location.

[0137] For example, the target key points of an object can be obtained by identifying an image including the first target object using an object tracking algorithm. For example, the object tracking algorithm can be implemented based on a pre-trained keypoint detection model. The number of target key points of the object can be determined, for example, based on the labels of the training samples set during the training process of the keypoint detection model. The keypoint detection model can include, for example, a deep learning model constructed based on coordinate value regression, a deep learning model constructed based on instance segmentation, or a deep learning model constructed based on heatmaps, etc., and this disclosure does not limit this. For example, the object tracking algorithm can also be implemented based on any existing method, and this disclosure does not limit this.

[0138] For example, an image including the first target object can be input into a deep learning model, which will then perform object detection and output the positions of key points of each object included in the first target object in the image, as well as the type of each object. For instance, if the first target object is a fingernail, the type of each object can be one of the following: thumb, index finger, middle finger, ring finger, and little finger.

[0139] For example, after obtaining the mesh model 1001 for each object and determining the display parameters 1002 based on the operation on the target window, the mesh model 1001 for each object can be adjusted based on the display parameters to obtain an adjusted mesh model 1003 for each object. For example, the display parameter 1002 may include a display length determined based on the adjustment length indicated by the aforementioned first length adjustment control. This embodiment can adjust the boundary points of the mesh model 1001 according to the display length to obtain the adjusted mesh model. The principle of adjusting the mesh model 1001 is the same as that described above for the operation on the target window. Figure 5 The principle of adjusting the mesh model 502 is similar and will not be repeated here.

[0140] For example, after obtaining the display parameters 1002 determined based on the operation on the target window, a second target image 1004 corresponding to the display parameters can also be determined. The second target image may include an image displayed on the object. For example, the display parameters 1002 may include a pattern type, and in this embodiment, a pattern including the pattern type can be used as the target image 1004. For example, the target image 1004 may be an image selected by the designer of the special effects material through a text selector, or it may be a pre-set texture selected by the designer of the special effects material. For example, the step of determining the target image 1004 and the step of adjusting the mesh model 1001 can be performed simultaneously, or they can be performed in any order. This embodiment of the present disclosure does not limit this.

[0141] After obtaining the second target image 1004 and the adjusted mesh model 1003, the target image can be attached to the surface of the adjusted model based on a predetermined mapping relationship, thereby obtaining an effect image 1005 on the object displaying the target image (e.g., displaying a pattern included in the target image). The principle of attaching the target image to the surface of the adjusted model in this embodiment is similar to the principle of attaching a predetermined image to the surface of the adjusted model in the previous embodiment, and will not be repeated here.

[0142] For example, the distribution of the mapping positions of the boundary points in the adjusted mesh model 1003 on the target image 1004 can be as follows: Figure 10 The points marked on the target image 1004 are not limited in this embodiment.

[0143] It is understandable that the display parameters can be the same or different for different types of objects. The above description only describes the principle of displaying an image with a pattern that matches the display parameters on a single type of object. However, the principle of displaying an image with a pattern that matches the display parameters corresponding to the type of object on different types of objects is similar.

[0144] This embodiment of the disclosure achieves the effect of displaying patterns from images onto objects by using a predetermined mapping relationship to attach images to a mesh model constructed based on the key points of the object. Compared to methods that construct masks and bind the object's structure to a 3D model, this method allows the displayed patterns to accurately follow the object's movement, improving the realism of trying on virtual items on the object. This is because the image-attaching technique based on mapping relationships renders images on the object in real time based on the position of the object's key points and the predetermined mapping relationship, and the predetermined mapping relationship does not change as the object moves.

[0145] Figure 11 The illustration schematically shows a principle diagram for determining target key points for each object according to at least one embodiment of the present disclosure.

[0146] In at least one embodiment of this disclosure, multiple keypoints of each detected object can be used as initial keypoints (also referred to herein as first keypoints). Subsequently, the position of the first keypoints is adjusted according to actual needs, and the adjusted keypoints are used as target keypoints.

[0147] For example, the positions of multiple first key points can be adjusted by expanding the initial area enclosed by multiple first key points to obtain the target key point.

[0148] like Figure 11As shown, in one embodiment, for each object 1101, an initial region 1102 can be determined based on multiple first key points of each object. For example, based on the positional relationship of multiple first key points, a line can be drawn between two adjacent first key points, and the area enclosed by the line can be used as the initial region.

[0149] After determining the initial region 1102, the initial region can be expanded to obtain the expanded region 1103. After obtaining the expanded region 1103, for example, the target key points for each object can be determined based on the multiple second key points on the boundary of the expanded region 1103 that correspond to the multiple first key points.

[0150] For example, multiple second keypoints on the boundary of the expanded region, corresponding to multiple first keypoints respectively, can be used as target keypoints for each object. Figure 11 As shown, if multiple first key points include key points 1 to 8, then the determined target key points may include eight second key points numbered 1' to 8' respectively.

[0151] In at least one embodiment of this disclosure, the image displayed on the object may include, for example, pixels corresponding one-to-one with the plurality of target key points, and the correspondence between the key points and the pixels may be the aforementioned predetermined mapping relationship. The image displayed on the object is a partial image of the area where the pattern is located in the entire image, and this partial image may be located in the central area of ​​the entire image including it, and other parts of the entire image other than the area where the pattern is located may be transparent, for example. Thus, when the image is overlaid onto the surface of the mesh model constructed based on the target key points, the pattern is displayed only in the central area of ​​the surface of the mesh model, and the edges of the image superimposed on the object are not affected by the angular boundaries of the mesh model. If the edges of the area where the pattern is located are smooth and without sharp edges, the image displayed on the object is also smooth. In this way, the image displayed on the object can have a smooth and natural effect, which can improve the realism of virtual objects being tried on and enhance the user experience of special effects materials.

[0152] Figure 12 The illustration schematically shows a principle diagram of determining an initial region for each object according to at least one embodiment of the present disclosure.

[0153] According to embodiments of this disclosure, when determining an initial region based on multiple first key points, the positions of these first key points can be appropriately adjusted firstly, and the initial region for each object can be determined based on the adjusted positions of the key points. Adjusting the positions of the first key points can, for example, aim to improve the aesthetic appeal of the object when trying on virtual items. For instance, in the case of a fingernail, the initial region for the fingernail can be appropriately lengthened by adjusting the positions of the first key points, since manicures are usually slightly longer than the fingernail itself for aesthetic purposes. This embodiment can, for example, improve the aesthetic appeal of the object after displaying an image on it, thus enhancing the user experience of special effects materials, and further increasing the click-through rate of products corresponding to the displayed image.

[0154] like Figure 12 As shown, each object 1201 may include a root and a top, and the detected multiple first keypoints of each object may include multiple root keypoints and multiple top keypoints. For example, if the multiple first keypoints of each object include, for example, Figure 15 Of the eight key points shown, the five key points numbered 1 to 5 are the top key points, and the three key points numbered 6 to 8 are the root key points. For example, the root and the top are relative to the growth direction of the object. As the object grows, the end whose position remains unchanged is the root, and the end whose position changes is the top.

[0155] For example, when adjusting the positions of multiple first keypoints, you can adjust only the positions of multiple top keypoints to adjust the length of the object displayed in the image to a certain extent. Depending on the actual needs, you can also adjust only the positions of multiple root keypoints, or adjust the positions of multiple top keypoints and multiple root keypoints simultaneously. The specific adjustment principle is similar to that of adjusting only the positions of multiple top keypoints. The following explanation uses adjusting only the positions of the top keypoints as an example to illustrate the adjustment principle.

[0156] For example, we can first determine the third distance between the root center point among multiple root keypoints and the top center point among multiple top keypoints. The third distance between these two center points can be determined based on the coordinates of the top center point and the root center point. Figure 11 As shown, the root center point can be the root keypoint numbered 7, the top center point can be the top keypoint numbered 3, and the third distance can be, for example, as shown below. Figure 11 The length represented by the dashed line connecting keypoint number 3 and keypoint number 7.

[0157] For example, after obtaining this third distance, multiple top keypoints can be adjusted based on at least this third distance. For example, a predetermined percentage of this third distance can be determined as the distance by which the multiple top keypoints are adjusted along the object's growth direction. For example, the object's growth direction can be... Figure 11 The Y-axis is shown as indicating the direction.

[0158] For example, after obtaining the third distance, the positions of keypoints other than the two edge keypoints among the multiple top keypoints can be adjusted based on this third distance. This is because, for an object (such as a fingernail), fixing the positions of the edge keypoints among the multiple top keypoints and adjusting only the positions of the other keypoints among the multiple top keypoints can make the shape of the object determined based on the adjusted keypoints more aesthetically pleasing, and thus improve the aesthetics after the image is displayed on the object. For example, the distance that these other keypoints are adjusted in the growth direction of the object can be determined by a predetermined proportion of the third distance.

[0159] After adjusting the positions of the key points, the initial region of object 1201 can be determined based on the adjusted key points and the key points whose positions have not been adjusted. For example, based on the positional relationship between the adjusted key points and the key points whose positions have not been adjusted, lines can be drawn between adjacent key points, and the area enclosed by the lines can be used as the initial region.

[0160] For example, when adjusting the top keypoint, in addition to adjusting the position of the top keypoint in the growth direction of the object, the position of the top keypoint can also be adjusted in the direction perpendicular to the growth direction. In this way, the shape of the object determined by the adjusted keypoint can be smoother and more beautiful, thereby improving the aesthetics of the image after it is displayed on the object.

[0161] For example, in addition to determining the third distance between the root center point and the top center point, a fourth distance can also be determined between two edge points among multiple top keypoints. For instance, the fourth distance between two edge points can be determined based on their coordinate values. Figure 12 As shown, the two edge points can include the top keypoint numbered 1 and the top keypoint numbered 5, and the fourth distance can be as follows: Figure 11 The length represented by the dashed line connecting the top keypoint numbered 1 and the top keypoint numbered 5.

[0162] For example, if the position of the top keypoint in the growth direction of the object is adjusted based on the third distance so that the appearance of the object represented by the adjusted keypoint is elongated, the position of the top keypoint in the direction perpendicular to the growth direction can be adjusted based on the fourth distance so that the adjusted keypoints converge towards the top center point, thereby making the appearance of the object represented by the adjusted keypoints more slender and smooth, thus improving the aesthetics of the image after it is displayed on the object.

[0163] For example, the position of the top center point (e.g., the top keypoint numbered 3) in the object's growth direction can be adjusted based on the third distance. Based on the third and fourth distances, the positions of the remaining keypoints (excluding the top center point and the two edge points) in the object's growth direction and in the direction perpendicular to the growth direction can be adjusted.

[0164] In at least one embodiment of this disclosure, the remaining keypoints can be adjusted based on a fourth distance, such as... Figure 11 As shown in the X-axis position, based on the third distance, adjust the remaining keypoints as follows: Figure 11 The position along the Y-axis is shown. For example, the X-axis can be parallel to the line connecting the top center point and the root center point, and the Y-axis can be parallel to the line connecting the two edge points.

[0165] For example, the top center point is the top key point numbered 3. In this embodiment, the position of the top center point can be adjusted along the extension of the line connecting the top key point numbered 7 and the top key point numbered 3, so that the distance between the adjusted top center point and the root center point is a first predetermined multiple of the aforementioned determined third distance. This first predetermined multiple can be any value greater than 1, such as 1.1, 1.15, etc., and this embodiment does not limit it.

[0166] For example, the remaining keypoints include top keypoint number 2 and top keypoint number 4. The positions of the remaining keypoints can be adjusted in the Y-axis direction so that the distance between the adjusted position of the remaining keypoint on the Y-axis and the position of the remaining keypoint on the Y-axis before adjustment is a second predetermined multiple of the aforementioned determined third distance. This second predetermined multiple can be any positive number, such as 0.008 or 0.005, which is less than the difference between the first predetermined multiple and 1. This embodiment of the present disclosure does not limit this. In the X-axis direction, top keypoint number 2 can be adjusted along the direction of the X-axis, and top keypoint number 4 can be adjusted in the opposite direction of the direction of the X-axis so that the distance between the adjusted position of the remaining keypoint on the X-axis and the position of the remaining keypoint on the X-axis before adjustment is a third predetermined multiple of the aforementioned determined fourth distance. This third predetermined multiple can be, for example, a small positive number, such as 0.005 or 0.01. This third predetermined multiple can be equal to or different from the second predetermined multiple. This embodiment of the present disclosure does not limit the value of the third predetermined multiple.

[0167] like Figure 12 As shown, the initial region enclosed by the adjusted keypoints and the unadjusted keypoints can be region 1202. From Figure 12 As can be seen, the initial region is more slender than the shape of the object. Therefore, the aesthetics of the image displayed on the object after subsequent processing based on this initial region will be improved to a certain extent.

[0168] In at least one embodiment of this disclosure, a first interface displays a target video frame, which includes a first target object. The target video frame can be, for example, any video frame from a video segment captured for the first target object. For instance, during the acquisition of the video segment, an image can be displayed on the first target object included in each video frame.

[0169] For example, when displaying an image on the first target object in the target video frame, the target key points of the objects included in the first target object in the target video frame can be determined based on the target key points of the objects in the first target object determined in the previous video frame. This makes the transition between the image displayed on the first target object in the first interface and the image displayed on the first target object in the target video frame smoother, thereby avoiding the position jitter of the displayed image caused by the change in the position of the first target object during video frame switching, which helps to improve the display effect of the displayed image and improve the user experience.

[0170] For example, when there are multiple preceding video frames for the target video frame, the first client application can cache at least two sets of target keypoints determined based on the first target object included in at least two preceding video frames. Each set of target keypoints includes multiple keypoints, which constitute a keypoint group. When determining the target keypoints for each object in the target video frame, multiple second keypoints in the expanded region corresponding to each object can be compared one-to-one with multiple keypoints in each cached keypoint group according to the correspondence between keypoints, to determine the offset of the multiple second keypoints relative to the keypoints included in each keypoint group. For example, the average, maximum or minimum value, or median of the multiple offsets obtained by comparing the keypoints one-to-one can be used as the offset of the multiple second keypoints relative to the keypoints included in each keypoint group.

[0171] For example, if the offsets of multiple second keypoints relative to keypoints in at least two cached keypoint groups are all greater than or equal to a predetermined offset, it indicates that the position of the first target object has changed significantly, and these multiple second keypoints can be used as target keypoints.

[0172] For example, if the offsets of multiple second keypoints relative to the keypoints included in at least one cached keypoint group are less than a predetermined offset, it indicates that the position of the first target object in the target video frame has changed little compared to the position of the first target object in a previous video frame. In this case, a target keypoint group can be determined from at least one keypoint group, and the positions of the multiple second keypoints can be adjusted based on the positions of the keypoints included in the target keypoint group. The adjusted multiple second keypoints are then used as multiple target keypoints. The target keypoint group can be the keypoint group whose offsets of the included keypoints are the smallest compared to the multiple second keypoints. That is, the offsets of the multiple second keypoint groups relative to the keypoints included in the target keypoint group are less than the offsets of the multiple second keypoint groups relative to the keypoints included in other keypoint groups.

[0173] For example, interpolation can be used to adjust the positions of multiple second keypoints. For instance, a first weight can be assigned to multiple second keypoints, and a second weight can be assigned to the keypoints in the target keypoint group. Based on the first and second weights, the positions of the second keypoints and the corresponding keypoints in the target keypoint group are weighted, and the weighted position is used as the adjusted position of the second keypoint. For example, the sum of the first and second weights is 1, and the values ​​of the first and second weights can be set according to actual needs; this embodiment does not limit this.

[0174] For example, at least two cached keypoint groups can be updated based on target keypoints determined for the target video frame. For instance, after adjusting the positions of multiple second keypoints based on the keypoints included in the target keypoint group, the target keypoint group can be updated to a keypoint group consisting of the determined multiple target keypoints.

[0175] For example, when multiple second keypoints are used as target keypoints, any one of the cached keypoint groups can be updated to a keypoint group composed of multiple second keypoints. Alternatively, the playback times of at least two preceding video frames corresponding to at least two keypoint groups can be determined first, and the keypoint group corresponding to the video frame with the earliest playback time can be updated to a keypoint group composed of multiple second keypoints. It is understood that the above-described mechanism for updating cached keypoint groups is merely an example to facilitate understanding of this disclosure, and this disclosure does not limit its scope.

[0176] For example, the keypoint corresponding to the second keypoint in the cached keypoint group is a keypoint with the same number as the second keypoint. The numbering rule for the keypoints of the first target object is the same for different video frames.

[0177] Figure 13 The illustration schematically shows a principle diagram of displaying an image determined based on an operation on a target window on a first target object according to at least another embodiment of the present disclosure.

[0178] In one embodiment, the first target object can be a fingernail, and the image displayed on the fingernail can include a nail art pattern. In this embodiment, the process of displaying the image on the first target object may include, for example, steps S1310 to S1360.

[0179] Step S1310: Obtain the detection result of the specified type of finger. This detection result can be the detection result of the fingernail, which includes the specified type of finger. The detection result includes multiple key points of the fingernail, for example, it can include eight or more key points; this embodiment of the present disclosure does not limit this. For example, the object tracking algorithm described above can be used to obtain the detection result. This detection result can be obtained, for example, by a server supporting the operation of the first client application processing an image including the specified type of finger, or by a terminal device running the first client application processing the image; this embodiment of the present disclosure does not limit this. The specified type can include any one or more types of the thumb, index finger, middle finger, ring finger, and little finger. For example, the specified type can be the type of finger targeted by the target window.

[0180] Step S1320: Adjust the detection points on the nail. For example, a method similar to... Figure 12The principle described is similar to that used to adjust the position of the top point in the nail's detection points (also known as key points), which will not be elaborated upon here.

[0181] Step S1330: Expand the initial area enclosed by the detection points to obtain the expanded area. For example, this can be achieved by using... Figure 11 The principle described is similar to that used to expand the initial area enclosed by the detection points, and will not be elaborated further here.

[0182] Step S1340: Adjust the key points according to the adjustment length. For example, the boundary points of the expanded region can be adjusted according to the adjustment length indicated by the first length adjustment control in the target window. Alternatively, between steps S1330 and S1340, inter-frame smoothing can be performed on the boundary points of the expanded region, i.e., adjusting the boundary points of the expanded region based on the cached key point group. The boundary points adjusted based on the adjustment length are the target key points obtained from the inter-frame smoothing process.

[0183] Step S1350: Generate a nail mesh model based on the adjusted keypoints. For example, the center point of the nail or the midpoint between the top center point and the root center point can be used as a reference point. Connect the points of each adjusted keypoint to this reference point, and connect the points of adjacent keypoints among the adjusted keypoints to construct a mesh model including multiple faces. Each face can be, for example, a triangular face formed by connecting three points.

[0184] In step S1360, based on a predetermined UV mapping relationship, the texture map (i.e., the image determined based on the operation on the target window) is applied to the surface of the mesh model to obtain the effect of displaying nail art patterns on the nail.

[0185] For example, to improve the efficiency of overlaying nail art patterns, a nail mesh model can be generated based on the boundary points of the expanded region or the target key points obtained through inter-frame smoothing before receiving an operation on the target window, i.e., before executing operation S1340. After detecting an operation on the first length adjustment control in the target window, step S1340 is then executed. In this embodiment, the key points adjusted in step S1340 are the vertices on the boundary lines of the mesh model.

[0186] Based on the image processing method provided in at least one embodiment of this disclosure, at least one embodiment of this disclosure also provides an image processing apparatus. The following will be combined with... Figure 14 The image processing apparatus is described in detail.

[0187] Figure 14 The schematic diagram illustrates a structural block diagram of an image processing apparatus provided in at least one embodiment of the present disclosure.

[0188] like Figure 14 As shown, the image processing apparatus 1400 of this embodiment includes a first display module 1410 and a second display module 1420. For example, these units or modules can be implemented by hardware (e.g., circuit) modules or software modules, etc. The following embodiments are similar and will not be repeated. For example, these units or modules can be implemented by a central processing unit (CPU), a general-purpose graphics processor (GPGPU), a graphics processing unit (GPU), a tensor processor (TPU), a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions.

[0189] The first display module 1410 is configured to display a target window in the first interface in response to an operation on a first target object in the first interface. For example, the first display module 1410 can be configured to perform step S210 described above. The specific implementation principle can be found in the relevant description of step S210, and will not be repeated here.

[0190] The second display module 1420 is configured to display a pattern determined based on the operation on the target window on the first target object in response to an operation on the target window. For example, the second display module 1420 can be configured to perform step S220 described above; its specific implementation principle can be found in the relevant description of step S220, and will not be repeated here.

[0191] In at least one embodiment of this disclosure, the first target object includes at least two objects belonging to at least two types. The first display module 1410 may, for example, include a first identifier display submodule and a page display submodule. The first identifier display submodule is configured to display at least two first identifier information corresponding to at least two types in a first interface in response to an operation on the first target object. The page display submodule is configured to display a target window for a first type in the first interface in response to a selection operation on first information among the at least two first identifier information, where the first type corresponds to the first information. The second display module 1420 may, for example, be configured to display an image determined based on an operation on the target window on the first object belonging to the first type among the at least two objects.

[0192] In at least one embodiment of this disclosure, the second display module 1420 may also be configured to: display a predetermined image on the first target object in response to an operation on the first target object; and adjust the predetermined image displayed on at least a portion of the first target object to an image determined based on the operation on the target window in response to an operation on the target window.

[0193] In at least one embodiment of this disclosure, the target window includes a target control, and operations on the target window include operations on the target control; the target control includes a first length adjustment control. The second display module 1420 described above can be configured such that, in response to an operation on the first length adjustment control, the length of a pattern comprising a predetermined image displayed on at least a portion of the object changes along the growth direction of the first target object, according to an adjustment length indicated by the first length adjustment control.

[0194] In at least one embodiment of this disclosure, the target window includes a target control, and operations on the target window include operations on the target control; the target control includes an image selection control. The second display module 1420 described above may include an image display submodule and an image replacement submodule. The image display submodule is configured to display a plurality of candidate images in response to an operation on the image selection control. The image replacement submodule is configured to replace a predetermined image displayed on at least a portion of the objects with the target image in response to a selection operation on a target image among the plurality of candidate images.

[0195] In at least one embodiment of this disclosure, the image processing apparatus 1400 may further include, for example, a hiding module configured to hide an image displayed on a second object of a second type among at least two objects in response to a deletion operation of a second information in at least two first identification information.

[0196] In at least one embodiment of this disclosure, the target window includes a target control, which includes an object type selection control. The image processing apparatus 1400 described above may also include, for example, an identifier display module. The identifier display module is configured to display at least two second identifier information corresponding to at least two types, respectively, in response to an operation on the object type selection control in the target window for a first type. The second display module 1420 described above may also be configured to display an image on a second object having the same display parameters as the image displayed on the first object, in response to a selection operation on third information corresponding to a second type among the at least two second identifier information.

[0197] In at least one embodiment of this disclosure, the first target object includes at least one of the following: fingernails and toenails.

[0198] In at least one embodiment of this disclosure, the image processing apparatus 1400 may further include, for example, a material determination module and a third display module. The material determination module is configured to obtain special effects material for the first target object in response to a determination operation of an image displayed on a first target object, the special effects material including the image displayed on the first target object. The third display module is configured to display the image included in the special effects material on a second target object in the second interface in response to a selection operation of special effects material displayed in a second interface.

[0199] In at least one embodiment of this disclosure, the image processing apparatus 1400 may further include a fourth display module, configured to display a second length adjustment control in the second interface in response to a selection operation of special effects material displayed in the second interface. The third display module may also be configured to, in response to an operation on the second length adjustment control, change the length of a pattern in the image displayed on the second target object along the growth direction of the second target object according to the adjustment length indicated by the second length adjustment control.

[0200] In at least one embodiment of this disclosure, the second display module 1420 may include, for example, a length determination submodule, a first model adjustment submodule, and a first mapping submodule. The length determination submodule is configured to determine a real-time adjustment length indicated by the first length adjustment control in response to an operation on the first length adjustment control. The first model adjustment submodule is configured to adjust the boundary points included in the mesh model for each of at least a portion of the objects based on the real-time adjustment length, to obtain an adjusted mesh model. The first mapping submodule is configured to attach a predetermined image to the surface of the adjusted mesh model based on a predetermined mapping relationship, such that the length of the pattern displayed on each object in the growth direction of each object changes with the adjustment length indicated by the first length adjustment control.

[0201] In at least one embodiment of this disclosure, the mesh model includes multiple boundary point groups, each boundary point group including a root point and a top point with a corresponding relationship. The aforementioned first model adjustment submodule may include a vector determination unit, a scale determination unit, and an adjustment unit. The vector determination unit is configured to determine an offset reference vector based on the root point and top point of the target group in the multiple boundary point groups in response to a real-time adjustment length greater than 1; the root point of the target group is the center point among the multiple root points, and the top point of the target group is the center point among the multiple top points. The scale determination unit is configured to determine an adjustment scale for each boundary point group based on a first distance between the root point and the top point of each boundary point group and a second distance between the root point and the top point of the target group. The adjustment unit is configured to determine the adjusted position of the top point of each boundary point group based on the adjustment scale for each boundary point group, the real-time adjustment length, and the offset reference vector.

[0202] In at least one embodiment of this disclosure, the image processing apparatus 1400 may further include, for example, a model acquisition module. This model acquisition module is configured to acquire a mesh model for each object, constructed based on target key points for each object, for each of at least a subset of the objects included in the first target object. The second display module 1420 may include, for example, a second model adjustment submodule, an image determination submodule, and a second mapping submodule. The second model adjustment submodule is configured to adjust the mesh model for each object based on an operation on the target window, obtaining an adjusted mesh model for each object. The image determination submodule is configured to determine a target image corresponding to an operation on the target window. The second mapping submodule is configured to attach the target image to the surface of the adjusted model for each object based on a predetermined mapping relationship.

[0203] In at least one embodiment of this disclosure, the target key points for each object are determined by: determining an initial region for each object based on a plurality of first key points for each object; expanding the initial region to obtain an expanded region; and determining the target key points for each object based on a plurality of second key points on the boundary of the expanded region that correspond to the plurality of first key points respectively, wherein the image displayed on each object is located in the central region of the target image, and the other parts of the target image other than the image displayed on each object are transparent parts.

[0204] In at least one embodiment of this disclosure, each object includes a root and a top; the plurality of first keypoints include a plurality of root keypoints and a plurality of top keypoints. Determining an initial region for each object based on the plurality of first keypoints includes: determining a third distance between a root center point among the plurality of root keypoints and a top center point among the plurality of top keypoints; adjusting the positions of other keypoints among the plurality of top keypoints, excluding two edge points, based at least on the third distance, to obtain adjusted keypoints; and determining the initial region for each object based on the adjusted keypoints, the two edge points, and the plurality of root keypoints.

[0205] In at least one embodiment of this disclosure, determining an initial region for each object based on a plurality of first keypoints of each object further includes determining a fourth distance between two edge points. The aforementioned adjustment of the positions of other keypoints among the plurality of top keypoints, excluding the two edge points, based at least on a third distance, to obtain adjusted keypoints includes: adjusting the position of the top center point in the growth direction of each object based on the third distance; and adjusting the positions of the remaining keypoints, excluding the top center point, in the growth direction of each object, and the positions of the remaining keypoints in a direction perpendicular to the growth direction, based on the third and fourth distances.

[0206] In at least one embodiment of this disclosure, determining target key points for each object based on multiple second key points corresponding to multiple first key points on the boundary of the expanded region includes: in response to the offset of multiple second key points relative to key points included in at least one of at least two predetermined key point groups being less than a predetermined offset, adjusting the positions of multiple second key points based on the positions of key points included in the target key point group in the at least two key point groups to obtain multiple target key points for each object, wherein the offset of multiple second key points relative to key points in the target key point group is less than the offset of multiple second key points relative to key points in other key point groups besides the target key point group in the at least two key point groups; the at least two predetermined key point groups are determined based on first target objects included in at least two preceding video frames of the target video frame; the first interface displays the target video frame, and the target video frame includes the first target object.

[0207] It should be noted that, for clarity and brevity, this disclosure does not provide all the constituent units of the image processing apparatus 1400. To achieve the necessary functions of the image processing apparatus 1400, those skilled in the art can provide and configure other constituent units (not shown) according to specific needs, and this disclosure does not impose any limitations on this.

[0208] At least one embodiment of this disclosure also provides an electronic device, including: a processing device; a storage device including one or more computer program modules; wherein the one or more computer program modules are stored in the storage device and configured to be executed by the processing device, and the one or more computer program modules are used to implement the image processing method provided in any embodiment of this disclosure.

[0209] For example, the processing device may be a central processing unit (CPU), digital signal processor (DSP), image processor (GPU), general-purpose graphics processor (GPGPU), or other form of processing unit with data processing capabilities and / or instruction execution capabilities. It may be a general-purpose processor or a dedicated processor and may control other components in the electronic device to perform the desired functions.

[0210] For example, the storage device may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, which a processing device may execute to implement the functions (implemented by the processing device) in the embodiments of this disclosure and / or other desired functions, such as image processing methods. Various application programs and various data may also be stored in the computer-readable storage medium, such as key points of the detected first target object, the mesh model of the first target object, predetermined mapping relationships, etc.

[0211] The following is for reference. Figure 15 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 1500 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 15 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0212] like Figure 15 As shown, electronic device 1500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1502 or a program loaded from storage device 1508 into random access memory (RAM) 1503. The RAM 1503 also stores various programs and data required for the operation of electronic device 1500. The processing device 1501, ROM 1502, and RAM 1503 are interconnected via bus 1504. Input / output (I / O) interface 1505 is also connected to bus 1504.

[0213] Typically, the following devices can be connected to I / O interface 1505: input devices 1506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1509. Communication device 1509 allows electronic device 1500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 15 An electronic device 1500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0214] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1509, or installed from storage device 1508, or installed from ROM 1502. When the computer program is executed by processing device 1501, it performs the functions defined in the methods of embodiments of this disclosure.

[0215] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0216] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0217] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0218] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: display a target window in the first interface in response to an operation on a first target object in the first interface; and display an image on the first target object based on the operation on the target window in response to an operation on the target window.

[0219] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: display a target window in the first interface in response to an operation on a first target object in the first interface; and display an image on the first target object based on the operation on the target window in response to an operation on the target window.

[0220] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0221] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0222] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units or modules do not necessarily constitute a limitation on the unit or module itself.

[0223] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0224] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0225] According to one or more embodiments of this disclosure, Example 1 provides an image processing method, including:

[0226] In response to an operation on a first target object in the first interface, a target window is displayed in the first interface; and

[0227] In response to an operation on a target window, display an image on a first target object that is determined based on the operation on the target window.

[0228] According to one or more embodiments of this disclosure, Example 2 provides that the first target object in Example 1 includes at least two objects belonging to at least two types;

[0229] The target window is displayed in the first interface in Example 1, including:

[0230] In response to an operation on a first target object, at least two first identification information corresponding to at least two types are displayed in a first interface; and

[0231] In response to a selection operation of first information from at least two first identification information, a target window for a first type is displayed on a first interface, wherein the first type corresponds to the first information;

[0232] Displaying an image on the first target object based on the operation performed on the target window, as shown in Example 1, includes:

[0233] Display an image on the first object, which belongs to the first type among at least two objects, based on the operation performed on the target window.

[0234] According to one or more embodiments of this disclosure, Example 3 provides the method of Example 1 or Example 2, further comprising:

[0235] In response to an operation on a first target object, a predetermined image is displayed on the first target object; displaying an image on the first target object based on an operation on a target window in Example 1 or Example 2 includes:

[0236] In response to an operation on a target window, a predetermined image displayed on at least a portion of the objects in a first target object is adjusted to an image determined based on the operation on the target window.

[0237] According to one or more embodiments of this disclosure, Example 4 provides a target window in Example 3 that includes a target control, and operations on the target window include operations on the target control, the target control including a first length adjustment control;

[0238] The adjustment of a predetermined image displayed on at least a portion of the objects in the first target object to an image determined based on an operation on the target window in Example 3 includes:

[0239] In response to an operation of the first length adjustment control, the length of the pattern included in the predetermined image displayed on at least a portion of the object changes along the growth direction of the first target object, according to the adjustment length indicated by the first length adjustment control.

[0240] According to one or more embodiments of this disclosure, Example 5 provides a target window in Example 3 that includes a target control, and operations on the target window include operations on the target control; the target control includes an image selection control;

[0241] Example 3, adjusting a predetermined image displayed on at least a portion of the objects in a first target object to an image determined based on an operation performed on the target window, includes:

[0242] In response to an operation on the image selection control, display multiple alternative images; and

[0243] In response to a selection operation of a target image among a plurality of candidate images, a predetermined image displayed on at least a portion of the objects is replaced with the target image.

[0244] According to one or more embodiments of this disclosure, Example Six provides the method of Example Two, further comprising:

[0245] In response to a deletion operation on the second information in at least two first identification information, the image displayed on the second object belonging to the second type among at least two objects is hidden.

[0246] According to one or more embodiments of this disclosure, Example 7 provides a target window in Example 6 that includes a target control, the target control including an object type selection control;

[0247] The methods in Example 6 also include:

[0248] In response to an operation on an object type selection control in a target window for the first type, display at least two second identification information corresponding to at least two types respectively; and

[0249] In response to a selection operation of a third piece of information corresponding to a second type from at least two second identification information, an image with the same display parameters as the image displayed on the first object is displayed on the second object.

[0250] According to one or more embodiments of this disclosure, Example 8 provides that the first target object in Example 1 or Example 2 includes at least one of the following: fingernails, toenails.

[0251] According to one or more embodiments of this disclosure, Example Nine provides the method of Example One or Example Two, further comprising:

[0252] In response to a determination operation on an image displayed on a first target object, special effects material for the first target object is obtained, the special effects material including the image displayed on the first target object; and

[0253] In response to the selection operation of the special effects material displayed in the second interface, the image including the special effects material is displayed on the second target object in the second interface.

[0254] According to one or more embodiments of this disclosure, Example 10 provides the method of Example 9, further comprising:

[0255] In response to the selection of special effects materials displayed in the second interface, a second length adjustment control is displayed in the second interface;

[0256] In response to an operation on the second length adjustment control, the length of the pattern in the image displayed on the second target object in the growth direction of the second target object changes according to the adjustment length indicated by the second length adjustment control.

[0257] According to one or more embodiments of this disclosure, Example 11 provides that the pattern of a predetermined image displayed on at least a portion of the object in Example 4 varies in length along the growth direction of the first target object with respect to an adjustment length indicated by a first length adjustment control, including:

[0258] In response to an operation on the first length adjustment control, determine the real-time adjustment length indicated by the first length adjustment control;

[0259] The adjusted mesh model is obtained by adjusting the boundary points of the mesh model for each object in at least a portion of the objects based on real-time length adjustment; and

[0260] Based on a predetermined mapping relationship, a predetermined image is attached to the surface of the adjusted mesh model so that the length of the pattern displayed on each object in the growth direction of each object changes with the adjustment length indicated by the first length adjustment control.

[0261] According to one or more embodiments of this disclosure, Example Twelve provides that the mesh model in Example Eleven includes multiple boundary point groups, each boundary point group including a root point and a top point with a corresponding relationship;

[0262] Example 11, which adjusts the boundary points of the mesh model for each object in at least a portion of the objects based on real-time length adjustment, includes:

[0263] In response to real-time adjustments with a length greater than 1, the offset reference vector is determined based on the root point and top point included in the target group among multiple boundary point groups; the root point included in the target group is the center point among multiple root points, and the top point included in the target group is the center point among multiple top points.

[0264] Based on the first distance between the root point and the top point of each boundary point group and the second distance between the root point and the top point of the target group, determine the adjustment ratio for each boundary point group; and

[0265] Based on the adjustment ratio, real-time adjustment length, and offset reference vector for each boundary point group, the adjusted position of the top point included in each boundary point group is determined.

[0266] According to one or more embodiments of this disclosure, Example Thirteen provides the method of Example One or Example Two, further comprising:

[0267] For each object among at least a portion of the objects included in the first target object, obtain a mesh model for each object based on the target key points for each object;

[0268] Displaying an image on the first target object based on an operation on the target window in Example 1 or Example 2 includes:

[0269] Based on the operations performed on the target window, the mesh model for each object is adjusted to obtain the adjusted mesh model for each object.

[0270] Determine the target image corresponding to the operation performed on the target window; and

[0271] Based on a predetermined mapping relationship, the target image is fitted onto the surface of the adjusted model for each object.

[0272] According to one or more embodiments of this disclosure, Example Fourteen provides that the target key points for each object in Example Thirteen are determined in the following manner:

[0273] Based on multiple first key points for each object, determine the initial region for each object;

[0274] The initial region is expanded to obtain the expanded region; and

[0275] Based on multiple second key points corresponding to multiple first key points on the boundary of the expanded region, the target key points for each object are determined.

[0276] The image displayed on each object is located in the central region of the target image, and the rest of the target image, excluding the image displayed on each object, is transparent.

[0277] According to one or more embodiments of this disclosure, Example Fifteen provides that each object in Example Fourteen includes a root and a top; the plurality of first keypoints include a plurality of root keypoints and a plurality of top keypoints;

[0278] Example 14 determines the initial region for each object based on multiple first keypoints, including:

[0279] Determine the third distance between the root center point among multiple root keypoints and the top center point among multiple top keypoints;

[0280] Based at least on the third distance, adjust the positions of all keypoints except the two edge points among multiple top keypoints to obtain the adjusted keypoints; and

[0281] Based on the adjusted keypoints, two edge points, and multiple root keypoints, the initial region for each object is determined.

[0282] According to one or more embodiments of this disclosure, Example Sixteen provides, as in Example Fifteen, determining an initial region for each object based on multiple first key points, and further includes:

[0283] Determine the fourth distance between two edge points;

[0284] In Example 15, based at least on the third distance, the positions of the other keypoints among multiple top keypoints, excluding the two edge points, are adjusted to obtain the adjusted keypoints, including:

[0285] Based on the third distance, adjust the position of the top center point in the growth direction of each object; and

[0286] Based on the third and fourth distances, adjust the positions of the remaining keypoints (excluding the top center point) in each object's growth direction, as well as the positions of the remaining keypoints in the direction perpendicular to the growth direction.

[0287] According to one or more embodiments of this disclosure, Example Seventeen provides the determination of target key points for each object based on multiple second key points corresponding to multiple first key points on the boundary of the expanded region, as in Example Fourteen, including:

[0288] In response to the fact that the offset of multiple second keypoints relative to keypoints included in at least one of at least two predetermined keypoint groups is less than a predetermined offset, the positions of the multiple second keypoints are adjusted based on the positions of keypoints included in the target keypoint group among the at least two keypoint groups, thereby obtaining multiple target keypoints for each object.

[0289] Among them, the offset of multiple second key points relative to key points in the target key point group is less than the offset of multiple second key points relative to key points in other key point groups besides the target key point group in at least two key point groups; the at least two predetermined key point groups are determined based on the first target object included in at least two preceding video frames of the target video frame; the first interface displays the target video frame, and the target video frame includes the first target object.

[0290] According to one or more embodiments of this disclosure, Example Eighteen provides an image processing apparatus, comprising:

[0291] The first display module is configured to display a target window in the first interface in response to an operation on a first target object in the first interface; and

[0292] The second display module is configured to display an image on the first target object based on the operation on the target window in response to an operation on the target window.

[0293] According to one or more embodiments of this disclosure, Example Nineteen provides an electronic device, including:

[0294] Processing device; and

[0295] Storage device, including one or more computer program instructions;

[0296] The one or more computer program instructions are executed by the processing device to perform the image processing method provided in at least one embodiment of the present disclosure.

[0297] According to one or more embodiments of the present disclosure, Example 20 provides a computer-readable storage medium that non-transitory stores computer-readable instructions, wherein the computer-readable instructions, when executed by a processor, implement the image processing method provided in at least one embodiment of the present disclosure.

[0298] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0299] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0300] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, comprising: In response to an operation on a first target object in the first interface, a target window is displayed in the first interface; as well as In response to an operation on the target window, an image determined based on the operation on the target window is displayed on the first target object.

2. The method according to claim 1, wherein, The first target object includes at least two objects belonging to at least two types; Displaying the target window in the first interface includes: In response to an operation on the first target object, at least two first identification information corresponding to the at least two types are displayed on the first interface; and In response to a selection operation of the first information among the at least two first identification information, a target window for the first type is displayed in the first interface, the first type corresponding to the first information; Displaying an image on the first target object based on an operation performed on the target window includes: An image determined based on the operation performed on the target window is displayed on a first object of the first type among the at least two objects.

3. The method according to claim 1 or 2, further comprising: In response to an operation on the first target object, a predetermined image is displayed on the first target object; Displaying an image on the first target object based on an operation performed on the target window includes: In response to an operation on the target window, the predetermined image displayed on at least a portion of the objects in the first target object is adjusted to an image determined based on the operation on the target window.

4. The method according to claim 3, wherein, The target window includes a target control, and operations on the target window include operations on the target control; the target control includes a first length adjustment control. Adjusting the predetermined image displayed on at least a portion of the objects in the first target object to an image determined based on an operation targeting the target window includes: In response to an operation of the first length adjustment control, the length of the pattern included in the predetermined image displayed on the at least part of the object in the growth direction of the first target object changes according to the adjustment length indicated by the first length adjustment control.

5. The method according to claim 3, wherein, The target window includes a target control, and operations on the target window include operations on the target control; the target control includes an image selection control. Adjusting the predetermined image displayed on at least a portion of the objects in the first target object to an image determined based on an operation targeting the target window includes: In response to an operation on the image selection control, multiple alternative images are displayed; as well as In response to a selection operation of a target image among the plurality of candidate images, the predetermined image displayed on at least a portion of the objects is replaced with the target image.

6. The method according to claim 2, further comprising: In response to a deletion operation on the second information in the at least two first identification information, the image displayed on the second object belonging to the second type among the at least two objects is hidden.

7. The method according to claim 6, wherein, The target window includes a target control, which includes an object type selection control. The method further includes: In response to an operation on the object type selection control in the target window for the first type. Display at least two second identification information corresponding to each of the at least two types; and In response to a selection operation of a third piece of information corresponding to the second type among the at least two second identification information, an image with the same display parameters as the image displayed on the first object is displayed on the second object.

8. The method according to claim 1 or 2, wherein, The first target object includes at least one of the following: fingernails or toenails.

9. The method according to claim 1 or 2, further comprising: In response to a determination operation for an image displayed on the first target object, special effects material for the first target object is obtained, the special effects material including the image displayed on the first target object; as well as In response to a selection operation of the special effects material displayed in the second interface, the image including the special effects material is displayed on the second target object in the second interface.

10. The method of claim 9, further comprising: In response to a selection operation of the special effects material displayed in the second interface, a second length adjustment control is displayed in the second interface; In response to an operation of the second length adjustment control, the length of the pattern in the image displayed on the second target object in the growth direction of the second target object changes according to the adjustment length indicated by the second length adjustment control.

11. The method according to claim 4, wherein, The length of the pattern included in the predetermined image displayed on at least a portion of the object in the growth direction of the first target object changes with the adjustment length indicated by the first length adjustment control, including: In response to an operation on the first length adjustment control, determine the real-time adjustment length indicated by the first length adjustment control; Based on the real-time length adjustment, the boundary points included in the mesh model for each object in at least a portion of the objects are adjusted to obtain the adjusted mesh model; and Based on a predetermined mapping relationship, the predetermined image is attached to the surface of the adjusted mesh model, such that the length of the pattern displayed on each object in the growth direction of each object changes with the adjustment length indicated by the first length adjustment control.

12. The method according to claim 11, wherein, The mesh model includes multiple boundary point groups, and each boundary point group includes a root point and a top point with corresponding relationships. The adjustment of the boundary points included in the mesh model for each object in at least a portion of the objects based on the real-time adjustment length includes: In response to the real-time adjustment length being greater than 1, an offset reference vector is determined based on the root point and top point included in the target group among the plurality of boundary point groups; the root point included in the target group is the center point among the plurality of root points, and the top point included in the target group is the center point among the plurality of top points; Based on the first distance between the root point and the top point of each boundary point group and the second distance between the root point and the top point of the target group, an adjustment ratio is determined for each boundary point group; and Based on the adjustment ratio for each boundary point group, the real-time adjustment length, and the offset reference vector, the adjusted position of the top point included in each boundary point group is determined.

13. The method according to claim 1 or 2, further comprising: For each of the at least some objects included in the first target object, obtain a mesh model for each object based on the target key points for each object; Displaying an image on the first target object based on an operation performed on the target window includes: Based on the operations performed on the target window, the mesh model for each object is adjusted to obtain an adjusted mesh model for each object; Determine the target image corresponding to the operation performed on the target window; and Based on a predetermined mapping relationship, the target image is fitted onto the surface of the adjusted model for each object.

14. The method according to claim 13, wherein, The target key points for each object are determined in the following way: Based on multiple first key points of each object, an initial region is determined for each object; The initial region is expanded to obtain the expanded region; as well as Based on multiple second key points on the boundary of the expanded region that correspond to the multiple first key points, target key points are determined for each object. The image displayed on each object is located in the central region of the target image, and the other parts of the target image besides the image displayed on each object are transparent.

15. The method according to claim 14, wherein, Each object includes a root and a top; the plurality of first keypoints include a plurality of root keypoints and a plurality of top keypoints; The determination of an initial region for each object based on multiple first key points of each object includes: Determine the third distance between the root center point among the plurality of root key points and the top center point among the plurality of top key points; Based at least on the third distance, adjust the positions of the other keypoints among the plurality of top keypoints, excluding the two edge points, to obtain the adjusted keypoints; and Based on the adjusted key points, the two edge points, and the multiple root key points, an initial region is determined for each object.

16. The method according to claim 15, wherein, The step of determining the initial region for each object based on multiple first key points of each object further includes: Determine the fourth distance between the two edge points; The adjustment of the positions of the other key points among the plurality of top key points, excluding the two edge points, based at least on the third distance, to obtain the adjusted key points, includes: Based on the third distance, adjust the position of the top center point in the growth direction of each object; and Based on the third distance and the fourth distance, adjust the positions of the remaining key points (excluding the top center point) in the growth direction of each object, as well as the positions of the remaining key points in the direction perpendicular to the growth direction.

17. The method of claim 14, wherein, The determination of target key points for each object based on multiple second key points corresponding to the multiple first key points on the boundary of the expanded region includes: In response to the fact that the offset of the plurality of second keypoints relative to keypoints included in at least one of the at least two predetermined keypoint groups is less than a predetermined offset, the positions of the plurality of second keypoints are adjusted based on the positions of keypoints included in the target keypoint group among the at least two keypoint groups, thereby obtaining a plurality of target keypoints for each object. Wherein, the offset of the plurality of second key points relative to the key points in the target key point group is less than the offset of the plurality of second key points relative to the key points in other key point groups besides the target key point group in the at least two key point groups; the at least two predetermined key point groups are determined based on the first target object included in at least two preceding video frames of the target video frame; the first interface displays the target video frame, and the target video frame includes the first target object.

18. An image processing apparatus, comprising: The first display module is configured to display a target window in the first interface in response to an operation on a first target object in the first interface; as well as The second display module is configured to display an image on the first target object based on the operation on the target window in response to an operation on the target window.

19. An electronic device comprising: processor; as well as Memory, which includes one or more computer program instructions; The one or more computer program instructions are executed by the processor according to any one of claims 1 to 17.

20. A computer-readable storage medium for non-transitory storage of computer-readable instructions, wherein, The method described in any one of claims 1 to 17 is implemented when the computer-readable instructions are executed by a processor.

Images