Image editing processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image editing processing method and device, electronic equipment and a storage medium, and relates to the technical field of images. The image recognition module and the skeleton adjustment module are deployed at the client at the same time, the functions of the image recognition module and the skeleton adjustment module can be fused, and image data information obtained after the image recognition module processes a source image can be sent to the server to be processed. And the target skeleton information corresponding to the object in the source image processed by the server can be directly sent to the skeleton adjustment module, and the skeleton adjustment module displays the skeleton image of the object in the source image according to the target skeleton information. Compared with the prior art, the server does not need to send the target skeleton information corresponding to the object in the source image to the image recognition module, and then the image recognition module forwards the target skeleton information to the external skeleton adjustment module for processing, so that the interaction process is simplified, and the interaction efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of image technology, and more particularly, to an image editing and processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Currently, the generation of intelligent paintings is usually achieved by means of plugins such as Controlnet (control network) and Openpose eidtor (pose editing), so that users can generate pictures with a higher degree of controllability.

[0003] However, the existing Openpose eidtor is based on the input image, calls a third-party plugin, fine-tunes the bone data in the image, and outputs the adjusted bone data to Controlnet for image generation.

[0004] However, the function of Openpose eidtor in the above method is relatively single, and the use of a third-party plugin for processing will make data interaction more complex, resulting in lower efficiency of image processing. Summary of the Invention

[0005] The purpose of this application is to provide an image editing and processing method, apparatus, electronic device, and storage medium to improve the image processing method in the intelligent drawing scenario.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of this application provides an image editing and processing method, which is applied to a client in an image editing system; an image recognition module and a bone adjustment module are deployed on the client, and the method includes:

[0008] Performing conversion processing on a source image input by a user through the image recognition module to obtain image data information of the source image, and sending the image data information of the source image to a server, where at least one object is included in the source image;

[0009] Receiving, by the bone adjustment module, target bone information corresponding to the object in the source image sent by the server, and displaying a bone image of the object according to the target bone information corresponding to the object; the target bone information includes information on each bone key point of the object;

[0010] Obtaining, by the bone adjustment module, an adjustment instruction for the input bone image, generating adjusted bone information of the object; and sending the adjusted bone information to the server.

[0011] Second aspect, an embodiment of the present application further provides an image editing and processing method, which is applied to a server in an image editing system; the method includes:

[0012] Receiving the image data information of the source image sent by the image recognition module of the client, and parsing the image data information of the source image to obtain the source image, where at least one object is included in the source image;

[0013] Extracting the original bone information of the object from the source image;

[0014] Converting the original bone information of the object to obtain the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object;

[0015] Sending the target bone information corresponding to the object to the bone adjustment module of the client.

[0016] Third aspect, an embodiment of the present application further provides an image editing and processing device, which is applied to a client in an image editing system; an image recognition module and a bone adjustment module are deployed on the client;

[0017] The image recognition module is used to perform conversion processing on the source image input by the user to obtain the image information of the source image, and send the image data information of the source image to the server, where at least one object is included in the source image;

[0018] The bone adjustment module is used to receive the target bone information corresponding to the object in the source image sent by the server, and display the bone image of the object according to the target bone information corresponding to the object; the target bone information is used to represent the information of each bone key point of the object;

[0019] The bone adjustment module is used to obtain the adjustment instruction of the input bone image, generate the adjusted bone information of the object; and send the adjusted bone information to the server.

[0020] Fourth aspect, an embodiment of the present application further provides an image editing and processing device, which is applied to a server in an image editing system; the device includes: a parsing module, a processing module, a conversion module, and a sending module;

[0021] The parsing module is used to receive the image information of the source image sent by the image recognition module of the client, and parse the image information of the source image to obtain the source image, where at least one object is included in the source image;

[0022] The processing module is used to extract the original bone information of the object from the source image;

[0023] The conversion module is configured to convert the original skeleton information of the object to obtain the target skeleton information corresponding to the object; the target skeleton information includes information on each skeleton key point of the object.

[0024] The sending module is configured to send the target skeleton information corresponding to the object to the skeleton adjustment module of the client.

[0025] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the image editing processing method provided in the first aspect or the second aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the image editing processing method provided in the first aspect or the second aspect.

[0027] The beneficial effects of the present application are as follows:

[0028] The present application provides an image editing processing method, apparatus, electronic device, and storage medium. By deploying an image recognition module and a skeleton adjustment module on the client at the same time, the functions of the image recognition module and the skeleton adjustment module can be integrated. The image data information obtained after the image recognition module processes the source image can be sent to the server for processing, and the target skeleton information corresponding to the object in the source image obtained after the server processes it can be directly sent to the skeleton adjustment module. The skeleton adjustment module displays the skeleton image of the object in the source image according to the target skeleton information. Compared with the prior art, the server does not need to send the target skeleton information corresponding to the object in the source image to the image recognition module, and then the image recognition module forwards the target skeleton information to an external skeleton adjustment module for processing, which simplifies the interaction process and improves the interaction efficiency. Moreover, through the operation dynamic response ability provided by the client, the adjustment operation of the skeleton image can be performed in the skeleton adjustment module without relying on a third-party plugin, reducing the processing time consumed by the repeated transmission of data, thereby improving the efficiency of image processing.

[0029] Secondly, the server extracts the skeletal information of the object in the source image and converts the skeletal information of the object in the source image into a preset format to obtain the target skeletal information in the preset format and transmits it back to the skeletal adjustment module of the client. Since the client and the server perform data interaction based on a unified data format, the skeletal adjustment module can quickly read the relevant information of the skeleton from the target skeletal information according to the preset format, so as to generate and display the skeletal image of the object. Performing data interaction based on a unified data format can improve the efficiency of data transmission and parsing processing.

[0030] The server extracts the original skeletal information of the object from the source image and converts the original skeletal information according to a predefined data format to represent the original skeletal information in a preset format, making the data format of the obtained target skeletal information clearer and more concise. The skeletal adjustment module of the client can also obtain the relevant information of the skeleton from the target skeletal information according to the preset format and display the skeletal image. The client and the server perform data interaction with the target skeletal information in the preset format, improving the data interaction and processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.

[0032] Figure 1 It is a schematic diagram of the architecture of an intelligent drawing system provided by an embodiment of the present application;

[0033] Figure 2 It is a schematic flowchart of an image editing and processing method provided by an embodiment of the present application;

[0034] Figure 3 It is a schematic flowchart of another image editing and processing method provided by an embodiment of the present application;

[0035] Figure 4 It is a schematic flowchart of yet another image editing and processing method provided by an embodiment of the present application;

[0036] Figure 5 It is a schematic flowchart of another image editing and processing method provided by an embodiment of the present application;

[0037] Figure 6 It is a schematic diagram of the display of a skeletal image provided by an embodiment of the present application;

[0038] Figure 7A schematic flowchart of another image editing and processing method provided by an embodiment of the present application;

[0039] Figure 8 A schematic flowchart of an image editing and processing method provided by an embodiment of the present application;

[0040] Figure 9 A schematic flowchart of another image editing and processing method provided by an embodiment of the present application;

[0041] Figure 10 A schematic flowchart of another image editing and processing method provided by an embodiment of the present application;

[0042] Figure 11 A schematic diagram of an image editing and processing device provided by an embodiment of the present application;

[0043] Figure 12 A schematic diagram of another image editing and processing device provided by an embodiment of the present application;

[0044] Figure 13 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0045] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0046] In addition, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0047] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0048] Currently, AI painting mainly supports two main forms: text-to-image and image-to-image. However, there are still a large number of optional operations available to users when generating images. The mainstream AI painting products in China mainly target non-professional users, so the optional operations are limited and easy to understand. However, for professional art, the images generated by the limited options cannot fully meet the needs. Therefore, more options / models are needed to further control the generation of images. The use of plugins such as Controlnet (control model) and Openpose editor (pose editor) enables users to generate images with a higher degree of controllability.

[0049] However, on the one hand, the existing openpose editor plugin does not have the function of dynamic image adjustment. When processing an image, it is based on the input source image, calls a third-party plugin to process the object bones in the source image, and finally obtains the processed bone data returned by the third-party plugin and sends it to the Controlnet plugin for image generation. This results in the need for repeated data transmission on different platforms, with a long data transmission link, affecting the processing efficiency.

[0050] On the other hand, the existing use of the Controlnet model and openpose editor is a simple webui (user interface) interaction form. Originally, Controlnet and openpose editor are two plugins, and their integrated use requires a large number of additional operations by the user, with a very high user interaction threshold and a large learning cost.

[0051] Based on this, the image editing and processing method provided by this solution can complete the processing of image bone data within openpose editor by designing a dynamic editing function in openpose editor, without the need to call a third-party plugin. In addition, by deploying the client of openpose editor and Controlnet on one device, the integrated design of openpose editor and Controlnet can be achieved, improving the image processing efficiency. In addition, this solution defines a preset data format to represent bone data, unifying the data transmission format between the client and the server, making the conversion efficiency between data and images higher, and improving the simplicity of data transmission and the efficiency of data processing.

[0052] Figure 1 Schematic diagram of the architecture of an intelligent drawing system provided by an embodiment of the present application, as Figure 1As shown in the figure, the system may include: a client and a server. An image recognition module and a skeleton adjustment module may be deployed on the client. Among them, the image recognition module may include a Controlnet plug-in, and the skeleton adjustment module may include an openpose editor plug-in.

[0053] The image recognition module in the client may perform image format conversion on the source image input by the user, convert the source image into a data format, and send it to the server. The server then obtains the source image by parsing, and through a skeleton recognition algorithm, extracts the skeleton data of the object in the source image, and converts the skeleton data into data in a predefined format, and sends it to the skeleton adjustment module in the client. Thus, the skeleton adjustment module can extract relevant information of the skeleton from the skeleton data according to the predefined format, and display the skeleton image. And the skeleton adjustment module can generate adjusted skeleton data based on the skeleton adjustment operation input by the user, and finally export the adjusted skeleton data to the server. The server generates an image according to the adjusted skeleton data to generate an object image with the skeleton pose corresponding to the adjusted skeleton data.

[0054] Figure 2 It is a schematic flowchart of an image editing and processing method provided by an embodiment of this application, which can be applied to Figure 1 the client in the intelligent drawing system shown in the figure; the method may include:

[0055] S201. Perform conversion processing on the source image input by the user through the image recognition module to obtain the image data information of the source image, and send the image data information of the source image to the server, where at least one object is included in the source image.

[0056] Optionally, the user can enter the user interaction interface by logging in to the client, and thus can enter the image recognition module through the interaction entry of the image recognition module on the user interaction interface, upload the source image. The image recognition module can obtain the source image in response to the operation of the user inputting the source image, and perform conversion processing on the source image to obtain the image data information of the source image.

[0057] Among them, it may be to convert the source image into a string information in base64 format to obtain the image data information of the source image, that is, convert the image into data for transmission. The conversion of the source image can be implemented through js (JavaScript, a lightweight, interpreted or just-in-time compiled programming language with function priority) and the general api interface provided by the browser.

[0058] At least one object may be included in the source image, and the object may be any type of object such as a person, an animal, a building, etc.

[0059] Before sending the image data information of the source image to the server, the image recognition module can also display the image data information of the source image in a normalized image processing component. Among them, the image display can be based on the third-party library konvajs of the canvas (canvas component) (a 2D js framework library developed based on Canvas), or other third-party libraries or native code can be used for processing. The purpose of this step is to display and view the source image on the client according to the image data information of the source image before sending the image data information of the source image to the server, and preprocessing operations can be performed on the source image. Here, the preprocessing can refer to some necessary processing work for the source image, such as compressing super-large images. Of course, for different images or different scenarios, the preprocessing performed on the source image can be different. In some cases, the preprocessing can also be operations such as binarizing or inverting the image, so as to obtain the image data information of the preprocessed source image.

[0060] S202. Receive the target bone information corresponding to the object in the source image sent by the server through the bone adjustment module, and display the bone image of the object according to the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object.

[0061] After sending the image data information of the preprocessed source image to the server, the server can perform bone recognition on the object in the source image and send the obtained target bone information corresponding to the object to the client. Here, the object can refer to the object in the source image. In this embodiment and subsequent embodiments, the object recognized in the source image is a person, and one object is taken as an example. When different objects are processed, only the bone key points are different.

[0062] Optionally, the target bone information corresponding to the object in the source image recognized by the server can be sent to the bone adjustment module in the client, so that the bone adjustment module receives the target bone information and displays the bone image of the object based on the target bone information.

[0063] Among them, the target bone information includes the information of each bone key point of the object, so that the bone image of the object can be restored and displayed according to the information of the bone key points.

[0064] Here, since the bone adjustment module and the image recognition module are fusion-deployed on the client, the server can directly send the target bone information corresponding to the object in the source image recognized to the bone adjustment module, without first sending it to the image recognition module and then having the image recognition module send it to a third-party bone adjustment module, avoiding page switching between the image recognition module and the bone adjustment module and saving the interaction process.

[0065] S203. Obtain an adjustment instruction for the input bone image through the bone adjustment module, generate adjusted bone information of the object; and send the adjusted bone information to the server.

[0066] In some embodiments, the user can input an adjustment operation for the bone image through the interaction interface of the bone adjustment module to change the posture of the bone. The adjustment operation for the bone image can be achieved by moving the positions of the bone key points, and the purpose of the adjustment is to make the adjusted bone image have the required posture. For example, the posture of the bone in the bone image before adjustment is the standing posture of a person, while the adjusted one may be the running posture of a person, etc.

[0067] Optionally, based on the adjustment operation for the bone image, adjusted bone information of the object can be generated. The adjusted bone information is a partial correction of the bone information on the basis of the target bone information.

[0068] Optionally, the bone adjustment module can send the generated adjusted bone information of the object to the server, so that the server can generate an image according to the adjusted bone information. Among them, the object in the generated image has the bone posture corresponding to the adjusted bone information.

[0069] In summary, for the image editing and processing method provided in this embodiment, by deploying the image recognition module and the bone adjustment module on the client at the same time, the functions of the image recognition module and the bone adjustment module can be integrated. The image data information obtained by the image recognition module after processing the source image can be sent to the server for processing, and the target bone information corresponding to the object in the source image obtained after the server processes can be directly sent to the bone adjustment module. The bone adjustment module can display the bone image of the object in the source image according to the target bone information. Compared with the prior art, the server does not need to send the target bone information corresponding to the object in the source image to the image recognition module, and then the image recognition module forwards the target bone information to an external bone adjustment module for processing, which simplifies the interaction process and improves the interaction efficiency. Moreover, through the operation dynamic response ability provided by the client, the adjustment operation of the bone image can be performed in the bone adjustment module without the need to execute with the help of a third-party plug-in, reducing the processing time consumed by the repeated transmission of data, thereby improving the efficiency of image processing.

[0070] Optionally, the target bone information includes a target array; the target array includes: a first field and a second field. The field value of the first field includes: the coordinates of multiple bone key points arranged in sequence; the field value of the second field includes: the connection information of each bone key point.

[0071] In an implementable manner, this solution can define the representation of skeletal information in the form of a target array. Optionally, the target array may include a first field and a second field. The first field may represent the coordinate field of key points, and the second field may represent the connection information field of skeletal points.

[0072] Both the first field and the second field may include multiple field elements, and each field element corresponds to a field value. The field values in the first field may include: the coordinates of multiple skeletal key points arranged in sequence; that is, one field value in the first field represents the coordinates of one skeletal key point. The field values in the second field may include: the connection information of each skeletal key point; that is, one field value in the second field may represent one connection information, and one connection information is composed of the start point identifier and the end point identifier of the connection.

[0073] Exemplarily, the target array may be represented as follows:

[0074] {"keypoints":[[251,160],[254,234],[191,232],...],

[0075] "connects":[[1,2],[1,5],[2,3],[3,4],...]}

[0076] Among them, keypoints refers to the first field (the coordinate field of skeletal key points). [251,160] is the first field value in the first field, [254,234] is the second field value in the first field, and so on. Each field value is a specific coordinate value, representing the coordinates of a certain skeletal key point. In some embodiments, for different types of objects, the sorting of the skeletal key points of the object may be preset in advance, and then the coordinates of each skeletal key point are filled into the first field in sequence according to the sorting.

[0077] Connects refers to the second field (the connection information field of skeletal points). [1,2] is the first field value in the second field, [1,5] is the second field value in the second field. Each field value represents one connection information, and the two data in each field value respectively represent the identifier of the start skeletal point and the identifier of the end skeletal point of the connection. However, since the image abstract structure of the skeletal point connection is an undirected graph, the start point and the end point do not need to be distinguished.

[0078] Moreover, the identifier of the bone point corresponds to the sorting in the first field of the coordinates of the bone point. Taking the connection information [1, 2] as an example, 1 can refer to the first sorted bone key point in the first field, and 2 can refer to the second sorted bone key point in the first field. Then, the connection information [1, 2] can represent the connection between the first sorted bone key point and the second sorted bone key point in the first field.

[0079] Of course, the above definition method of the second field in the target array is only a possible illustration. In actual applications, identifiers can also be set for each bone key point in advance, so that the field values in the second field can be generated according to the identifiers of each bone key point and the connections between the bone key points.

[0080] Figure 3 It is a schematic flowchart of another image editing and processing method provided by an embodiment of the present application. Optionally, in step S202, displaying the bone image of the object according to the target bone information corresponding to the object may include:

[0081] S301. Obtain the coordinates of each bone key point of the object and the connection information between each bone key point from the target array.

[0082] Since the client and the server comply with a unified data format protocol, the bone adjustment module can interpret the received target bone information according to the data format protocol followed by the client and the server, and obtain the coordinates of each bone key point of the object and the connection information between each bone key point from the target bone information. That is, the coordinates of each bone key point of the object and the connection information between each bone key point can be obtained from the target array.

[0083] Optionally, it may be to respectively read each field value from the first field of the target array to obtain the coordinates of each bone key point; respectively read each field value from the second field of the target array, and obtain the connection information between each bone key point according to the starting bone point and the ending bone point of the connection identified in the field value.

[0084] S302. Display the bone image of the object according to the coordinates of each bone key point of the object, the connection information between each bone key point, the color information of each bone key point, and the connection color information of each bone key point.

[0085] In some embodiments, the bone adjustment module may further obtain the color information of each bone key point and the connection line color information of each bone key point. Thus, according to the coordinates of each bone key point, the positions of each bone point can be calibrated in the user interface. According to the connection line information between each bone key point, the calibrated bone key points are connected. At the same time, according to the color information of the bone key points, each bone key point is displayed in a set color, and according to the connection line color information of each bone key point, each connection line is displayed in a set color, so as to generate and display the bone image of the object. By setting the colors of the bone key points and the connection lines, it is convenient for the user to more clearly distinguish different bone points and connection lines, and it is convenient for the user to accurately locate the bone key points to be adjusted.

[0086] Figure 4 It is a schematic flowchart of another image editing and processing method provided by an embodiment of the present application. Optionally, in step S302, before displaying the bone image of the object according to the coordinates of each bone key point of the object, the connection line information between each bone key point, the color information of each bone key point, and the connection line color information of each bone key point, it may include:

[0087] S401. Generate the field value of the third field in the target array according to the preset color information of each bone key point. The field value of the third field includes: the color identifiers of each bone key point arranged in sequence; each field value in the third field corresponds one-to-one with each field value in the first field.

[0088] In some embodiments, the defined target array may further include a third field and a fourth field. The third field is used to represent the color information of the bone key points, and the fourth field is used to represent the connection line color information of the bone key points.

[0089] When the server side extracts the target bone information of the object from the source image, only the coordinate information of the bone key points and the connection line information of the bone key points are extracted, and the colors of the bone key points and the connection line colors can be defined by the client itself. Then, in the target bone information returned by the server to the bone adjustment module, the third field and the fourth field of the target array also exist, but the field values therein are empty. After receiving the target bone information, the bone adjustment module fills the corresponding field values into the third field and the fourth field to generate a complete target array.

[0090] Exemplarily, the complete target array may be as follows:

[0091] {"keypoints":[[251,160],[254,234],[191,232],...],

[0092] "connects": [[1, 2], [1, 5], [2, 3], [3, 4],...],

[0093] "keypoint_cids": [0, 1, 2, 3, 4,...],

[0094] "connect_cids": [0, 1, 2, 3, 4,...]}

[0095] Among them, keypoint_cids refers to the third field (the color field of the skeletal key points), and each field value in the third field can respectively correspond to different colors. For example, 0 represents white, 1 represents red, and 2 represents yellow.

[0096] Optionally, each field value in the third field can correspond one-to-one with each field value in the first field. That is, the first field value in the third field corresponds to the first field value in the first field. Then, it can be determined that the skeletal key point represented by the first field value in the first field has the color identified by the first field value in the third field. Assuming that the skeletal key point represented by the first field value in the first field is skeletal key point 1, and the color identified by the first field value in the third field is white, then, that is, skeletal key point 1 is white.

[0097] S402. Generate the field values of the fourth field in the target array according to the preset color information of the connections between the skeletal key points. The field values of the fourth field include: the color identifiers of the connections between the skeletal key points arranged in sequence; each field value in the fourth field corresponds one-to-one with each field value in the second field.

[0098] Similarly, connect_cids refers to the fourth field (the connection color field of the skeletal key points), and each field value in the fourth field can respectively correspond to different colors, and there may be the same field values between the third field and the fourth field.

[0099] Each field value in the fourth field can correspond one-to-one with each field value in the second field. That is, the first field value in the fourth field corresponds to the first field value in the second field. Then, assuming that the first field value in the second field represents the connection between skeletal key point 1 and skeletal key point 2, and the first field value in the fourth field represents yellow, then, it can be determined that the connection between skeletal key point 1 and skeletal key point 2 is yellow.

[0100] S403. Obtain the color information of each skeletal key point according to the field values of the third field in the target array.

[0101] Optionally, based on the above definition of the third field, the color information of each skeletal key point can be obtained according to the field values read from the third field.

[0102] S404. Obtain the connection line color information of each skeletal key point according to the field value of the fourth field in the target array.

[0103] Similarly, by reading the field value of the fourth field in the target array, the connection line color information of each skeletal key point can be obtained respectively.

[0104] Then, based on the coordinates and colors of the determined skeletal key points, the connection lines between the skeletal key points, and the connection line colors, a skeletal image of the object can be generated for display.

[0105] Optionally, in step S202, displaying the skeletal image of the object may include: displaying the skeletal image of the object in a transparent layer, and the transparent layer is located above the layer of the source image.

[0106] In this embodiment, a transparent layer can be added in the client, so that the skeletal image of the object can be displayed in the transparent layer, and the transparent layer is located above the layer of the source image. Thus, the skeletal image and the source image can be correspondingly displayed, saving the data forwarding time for transmitting the skeletal image to a third-party plugin for display.

[0107] Optionally, through layer processing in image software such as PS, the skeletal image and the source image can be split into two layers with the same layer size and completely overlapping, but the focuses of display and interaction are completely different.

[0108] Only the skeletal image is displayed in the transparent layer for displaying the skeletal image. The skeletal image itself is implemented by a canvas component to support event processing, mainly the support for the drag event.

[0109] Based on the above design, it is possible to support the display of the source image and the skeletal image in the same interaction scenario without affecting the adjustment of the skeletal image.

[0110] Figure 5 This is a schematic flowchart of another image editing and processing method provided by an embodiment of the present application. Optionally, in step S203, obtaining an adjustment instruction for the input skeletal image and generating adjusted skeletal information of the object may include:

[0111] S501. Receive an adjustment operation of a specified skeletal key point of the input object through a graphic interface provided by a canvas plugin, and obtain new coordinates of the specified skeletal key point.

[0112] Optionally, through the graphic interface provided by the canvas, an adjustment operation on a specified bone key point in the bone image of the object input by the user can be responded to, so as to obtain new coordinates of the specified bone key point. The specified bone key point can be any key point among the bone key points, and the specified bone key point is determined based on the bone pose adjustment requirement.

[0113] S502. According to the new coordinates of the specified bone key point, correct the coordinates of the corresponding bone key point in the target bone information corresponding to the object, and generate new bone information corresponding to the object.

[0114] Then, the new coordinates of the obtained specified bone key point can be used to replace the original coordinates of the specified bone key point in the target bone information of the object, so as to obtain new bone information corresponding to the object.

[0115] Of course, in some cases, it is also possible to correct the color of the bone key point and the color of the connection line. Similarly, the corrected color can be used to replace the original color in the target bone information.

[0116] The connection line of the bone key point is basically not adjusted, and the adjustment operation is usually performed on the position of the bone key point.

[0117] Figure 6 This is a schematic diagram of bone image display provided by an embodiment of the present application. Figure 6 In (a) is the bone image before adjustment. Figure 6 In (b) is the bone image after adjustment.

[0118] Optionally, in step S203, sending the adjusted bone information of the object to the server includes: sending the adjusted bone information to the server through the graphic interface provided by the canvas plugin.

[0119] In some embodiments, the bone adjustment module can send the adjusted bone information to the server based on the API provided by konvajs. And in some embodiments, the bone adjustment module can also export the adjusted bone image in the transparent layer of the canvas in the whole canvas through the capabilities provided by the image component. Among them, the adjusted bone image can be converted into data in base64 format for export, so as to import the whole adjusted bone information and the image information of the adjusted bone image into the server together.

[0120] In an implementable manner, the image recognition module in the client may also not receive the source image input by the user and send it to the server to extract the target bone information. Instead, the user can directly retrieve the required basic bone image from the bone image library through the control in the user interface. Thus, the bone adjustment module can directly perform adjustment operations on the basic bone image to generate the adjusted bone image, saving the data interaction between the client and the server.

[0121] In summary, for the image editing and processing method provided in this embodiment, by deploying the image recognition module and the bone adjustment module in the client at the same time, the functions of the image recognition module and the bone adjustment module can be integrated. The image data information after the source image is processed by the image recognition module can be sent to the server for processing, and the target bone information corresponding to the object in the source image obtained after the server processes can be directly sent to the bone adjustment module, and the bone adjustment module can display the bone image of the object in the source image according to the target bone information. Compared with the prior art, the server does not need to send the target bone information corresponding to the object in the source image to the image recognition module, and then the image recognition module forwards the target bone information to an external bone adjustment module for processing, simplifying the interaction process and improving the interaction efficiency. Moreover, through the operation dynamic response ability provided by the client, the adjustment operation of the bone image can be performed in the bone adjustment module without the need to execute with the aid of a third-party plug-in, reducing the processing time consumed by the repeated transmission of data, thereby improving the efficiency of image processing.

[0122] Secondly, the server extracts the bone information of the object in the source image and converts the bone information of the object in the source image according to a preset format to obtain the target bone information with the preset format and send it back to the bone adjustment module of the client. Since the client and the server perform data interaction based on a unified data format, the bone adjustment module can quickly read the relevant information of the bone from the target bone information according to the preset format, thereby generating and displaying the bone image of the object. Performing data interaction based on a unified data format can improve the efficiency of data transmission and parsing processing.

[0123] Figure 7 FIG. is a schematic flowchart of another image editing and processing method provided in an embodiment of the present application. Optionally, in step S201, performing conversion processing on the source image input by the user to obtain the image data information of the source image and sending the image data information of the source image to the server may include:

[0124] S701. Perform data format conversion on the source image to obtain the image data information of the source image.

[0125] For the specific implementation of this step, refer to the description in step S201, that is, convert the source image into a string information in base64 format to obtain the image data information of the source image.

[0126] S702. Obtain at least one piece of preprocessing parameter information corresponding to the input source image.

[0127] In some embodiments, the user can also input the preprocessing parameters for the source image through the preprocessing parameter upload entry provided by the image recognition module. The preprocessing here is different from the preprocessing process in step S201. The preprocessing parameter information includes, but is not limited to, parameters such as preprocessing algorithms, influence weights, guiding intensities, and image resolutions.

[0128] S703. Send the image data information of the source image and at least one piece of preprocessing parameter information corresponding to the source image to the server.

[0129] The image recognition module can send the obtained preprocessing parameters and the converted image data information of the source image to the server together. Thus, the server can restore the source image according to the image data information of the source image, perform preprocessing on the source image according to the preprocessing parameters, and perform bone information recognition based on the preprocessed source image to obtain the target bone information of the object.

[0130] Next, the method executed on the server side that interacts with the above client is described.

[0131] Figure 8 It is a schematic flowchart of an image editing and processing method provided by an embodiment of the present application, which is applied to a server in an image editing system; as Figure 8 shown, the method may include:

[0132] S801. Receive the image information of the source image sent by the image recognition module of the client, and parse the image information of the source image to obtain the source image, which includes at least one object.

[0133] Referring to the foregoing embodiments, the image information of the source image received by the server can be data in base64 format, and by parsing the image information of the source image, the source image can be restored. Among them, the source image may include at least one object, and the object here can be any type of object such as a person, an animal, or a building. And the number of objects is not limited to one, and multiple objects can be included simultaneously.

[0134] S802. Extract the original bone information of the object from the source image.

[0135] Optionally, in this embodiment, taking a person as an example of the object, a bone recognition model can be used to recognize the source image and extract the original bone information of the object. The extracted original bone information is just a bunch of data without a specific format, so it is difficult to read and process the data.

[0136] S803. Convert the original bone information of the object to obtain the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object.

[0137] In this embodiment, the original bone information of the extracted object can be converted according to a preset data format to obtain the target bone information corresponding to the object. Then, the target bone information is represented in a preset format. That is to say, this solution predefines a data format, so that the original bone information can be converted into an expression in a preset format, and the obtained target bone information is clearer and more concise.

[0138] Among them, the target bone information can include the information of each bone key point of the object; the bone key point can be a point preset from all bone points that can best represent the bone posture.

[0139] S804. Send the target bone information corresponding to the object to the bone adjustment module of the client.

[0140] Optionally, the server can send the target bone information corresponding to the object obtained after conversion to the bone adjustment module of the client, so that the bone adjustment module can restore and display the object bone image according to the target bone information.

[0141] In summary, the image editing and processing method provided in this embodiment extracts the original bone information of the object from the source image, converts the original bone information according to the pre-defined data format, so as to convert the original bone information into a preset format for representation, making the data format of the obtained target bone information clearer and more concise. The bone adjustment module of the client can also obtain the relevant information of the bone from the target bone information according to the preset format to display the bone image. The client and the server perform data interaction with the target bone information in a preset format, improving the data interaction and processing efficiency.

[0142] Optionally, in step S802, extracting the original bone information of the object from the source image may include: recognizing the source image, determining the positions of each bone key point of the object in the source image and the connections between each bone key point, and taking the positions of each bone key point and the connections between each bone key point as the original bone information.

[0143] A bone recognition algorithm can be used to recognize the object in the source image, and determine the positions of each bone key point of the object and the connections between the bone key points.

[0144] Generally, for different types of objects, the skeletal key points contained therein can be preset. For example, when the object is a person, it is usually defined that the skeletal key points of a person can include 18, such as: left eye, right eye, eyebrows, mouth, left shoulder, left axis, right leg joint, etc.

[0145] Then, according to the type of the object and the skeletal key points corresponding to the object type, the positions of the respective skeletal key points of the object can be recognized. And when a certain skeletal key point does not exist, it is marked.

[0146] Figure 9 It is a schematic flowchart of another image editing and processing method provided by an embodiment of the present application. Optionally, step of converting the original skeletal information of the object to obtain the target skeletal information corresponding to the object may include:

[0147] S901. Determine the coordinates of each skeletal key point according to the positions of each skeletal key point.

[0148] Then, according to the positions of the respective skeletal key points recognized, the coordinates of each skeletal key point can be obtained. The coordinates here may refer to the coordinates in the image coordinate system corresponding to the source image.

[0149] S902. Establish a correspondence relationship between two skeletal key points connected to each other according to the connection lines between each skeletal key point.

[0150] And according to the connection lines between the respective skeletal key points recognized, two connected skeletal key points can be made to correspond one by one to construct a correspondence relationship.

[0151] S903. Generate the field values of each field in the target array according to the coordinates of each skeletal key point and the correspondence relationship between two skeletal key points connected to each other, and use the target array as the target skeletal information corresponding to the object.

[0152] Then, according to the coordinates of each skeletal key point and the correspondence relationship between two skeletal key points connected to each other, the field value of the first field and the field value of the second field in the target array can be generated respectively, so as to obtain the target skeletal information with a preset format.

[0153] Figure 10 It is a schematic flowchart of yet another image editing and processing method provided by an embodiment of the present application. Optionally, in step S903, generating the field values of each field in the target array according to the coordinates of each skeletal key point and the correspondence relationship between two skeletal key points connected to each other may include:

[0154] S1001. Generate the field value of the first field in the target array according to the coordinates of each skeletal key point. The field value of the first field includes the coordinates of multiple skeletal key points arranged in sequence.

[0155] Referring to the description of the first field in the target array in the above embodiments, the coordinates of each skeletal key point can be filled into the first field in sequence according to a preset order. The preset order here is not limited, and there is no specific requirement for the arrangement order of the coordinates of each skeletal key point in the first field. As long as the coordinates of each skeletal key point are filled once, and the coordinates of all skeletal key points of the object can be completely included in the first field.

[0156] S1002. Generate the field value of the second field in the target array according to the positions of the coordinates of each skeletal key point in the field value of the first field and the corresponding relationship between two connected skeletal key points.

[0157] According to the positions of the coordinates of each skeletal key point in the field value of the first field, the identifiers of each skeletal key point can be determined respectively. Then, according to the identifiers of each skeletal key point and the corresponding relationship between two connected skeletal key points, the field value of the second field in the target array can be generated. For a detailed description, refer to the explanation of the target array in the previous embodiments for understanding.

[0158] In summary, the image editing and processing method provided in this embodiment extracts the original skeletal information of the object from the source image, converts the original skeletal information according to a predefined data format, so as to convert the original skeletal information into a preset format for representation, making the data format of the obtained target skeletal information clearer and more concise. The skeletal adjustment module on the client can also obtain the relevant information of the skeleton from the target skeletal information according to the preset format for displaying the skeletal image. The client and the server perform data interaction with the target skeletal information in the preset format, improving the data interaction and processing efficiency.

[0159] The following describes the apparatus, device, storage medium, etc. for executing the image editing and processing method provided in this application. For the specific implementation process and technical effects, refer to the above, and will not be repeated below.

[0160] Figure 11 It is a schematic diagram of an image editing and processing apparatus provided in an embodiment of this application. The functions implemented by this image editing and processing apparatus correspond to the steps of the method executed by the above client. The apparatus may include an image recognition module 110 and a skeletal adjustment module 120 deployed on the client.

[0161] The image recognition module 110 is configured to receive the target bone information corresponding to the object in the source image sent by the server, and display the bone image of the object according to the target bone information corresponding to the object; the target bone information is used to represent the information of each bone key point of the object.

[0162] The bone adjustment module 120 is configured to receive the target bone information corresponding to the object in the source image sent by the server, and display the bone image of the object according to the target bone information corresponding to the object; the target bone information is used to represent the information of each bone key point of the object.

[0163] The bone adjustment module 120 is configured to obtain the adjustment instruction of the input bone image, generate the adjusted bone information of the object; and send the adjusted bone information to the server.

[0164] Optionally, the target bone information includes a target array, and the target array includes: a first field and a second field. The field value of the first field includes: the coordinates of multiple bone key points arranged in sequence; the field value of the second field includes: the connection information of each bone key point.

[0165] Optionally, the bone adjustment module 120 obtains the coordinates of each bone key point of the object and the connection information between each bone key point from the target array;

[0166] According to the coordinates of each bone key point of the object, the connection information between each bone key point, the color information of each bone key point, and the connection color information of each bone key point, display the bone image of the object.

[0167] Optionally, the bone adjustment module 120 is further configured to generate the field value of the third field in the target array according to the preset color information of each bone key point. The field value of the third field includes: the color identifiers of each bone key point arranged in sequence; each field value in the third field corresponds one-to-one with each field value in the first field;

[0168] Generate the field value of the fourth field in the target array according to the preset color information of the connection between each bone key point. The field value of the fourth field includes: the color identifiers of the connection between each bone key point arranged in sequence; each field value in the fourth field corresponds one-to-one with each field value in the second field;

[0169] Obtain the color information of each bone key point according to the field value of the third field in the target array;

[0170] Obtain the connection color information of each bone key point according to the field value of the fourth field in the target array.

[0171] Optionally, the bone adjustment module 120 is specifically configured to display the bone image of the object in a transparent layer, and the transparent layer is located above the layer of the source image.

[0172] Optionally, the bone adjustment module 120 is specifically configured to receive an adjustment operation of specified bone key points of an input object through a graphic interface provided by a canvas plug-in, and obtain new coordinates of the specified bone key points;

[0173] According to the new coordinates of the specified bone key points, correct the coordinates of the corresponding bone key points in the target bone information corresponding to the object, and generate new bone information corresponding to the object.

[0174] Optionally, the bone adjustment module 120 is specifically configured to send the adjusted bone information to the server through a graphic interface provided by a canvas plug-in.

[0175] Optionally, the image recognition module 110 is specifically configured to convert the data format of the source image to obtain image data information of the source image; obtain at least one preprocessing parameter information corresponding to the input source image; and send the image data information of the source image and at least one preprocessing parameter information corresponding to the source image to the server.

[0176] Figure 12 FIG. is a schematic diagram of another image editing and processing device provided by an embodiment of the present application. The functions implemented by the image editing and processing device correspond to the steps of the method executed on the server side. The device may include: a parsing module 210, a processing module 220, a conversion module 230, and a sending module 240;

[0177] The parsing module 210 is configured to receive the image information of the source image sent by the image recognition module of the client, and parse the image information of the source image to obtain the source image, where the source image includes at least one object;

[0178] The processing module 220 is configured to extract the original bone information of the object from the source image;

[0179] The conversion module 230 is configured to convert the original bone information of the object to obtain target bone information corresponding to the object, where the target bone information includes information of each bone key point of the object;

[0180] The sending module 240 is configured to send the target bone information corresponding to the object to the bone adjustment module of the client.

[0181] Optionally, the processing module 220 is specifically configured to recognize the source image, determine the positions of each bone key point of the object in the source image and the connection lines between each bone key point, and use the positions of each bone key point and the connection lines between each bone key point as the original bone information.

[0182] Optionally, the conversion module 230 is specifically configured to determine the coordinates of each bone key point according to the positions of each bone key point;

[0183] Establish the corresponding relationship between two skeleton key points connected to each other according to the connection lines between the skeleton key points;

[0184] Generate the field values of each field in the target array according to the coordinates of each skeleton key point and the corresponding relationship between two skeleton key points connected to each other, and use the target array as the target skeleton information corresponding to the object.

[0185] Optionally, the conversion module 230 is specifically configured to generate the field value of the first field in the target array according to the coordinates of each skeleton key point, and the field value of the first field includes: the coordinates of multiple skeleton key points arranged in sequence;

[0186] Generate the field value of the second field in the target array according to the position of the coordinates of each skeleton key point in the field value of the first field and the corresponding relationship between two skeleton key points connected to each other.

[0187] The above modules can be one or more integrated circuits configured to implement the above methods. For example: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), etc. Again, when a certain above module is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0188] The above modules can be connected or communicate with each other via wired connections or wireless connections. Wired connections can include metal cables, optical cables, hybrid cables, etc., or any combination thereof. Wireless connections can include connections in the form of LAN, WAN, Bluetooth, ZigBee, or NFC, etc., or any combination thereof. Two or more modules can be combined into a single module, and any one module can be divided into two or more units. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated in this application.

[0189] Figure 13A schematic structural diagram of an electronic device provided by an embodiment of the present application, including: a processor 801, a storage medium 802, and a bus 803. The storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device is a client, it is used to execute the image editing processing method executed by the client in the above embodiment. When the electronic device is a server, it is used to execute the image editing processing method executed by the server in the above embodiment. The processor 801 communicates with the storage medium 802 through the bus 803. The processor 801 executes the machine-readable instructions to perform the following steps:

[0190] The source image input by the user is converted through an image recognition module to obtain the image data information of the source image, and the image data information of the source image is sent to the server, where at least one object is included in the source image;

[0191] The target bone information corresponding to the object in the source image sent by the server is received through a bone adjustment module, and the bone image of the object is displayed according to the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object;

[0192] The adjustment instruction of the input bone image is obtained through the bone adjustment module to generate the adjusted bone information of the object; and the adjusted bone information is sent to the server.

[0193] In a feasible implementation, the target bone information includes a target array; the target array includes: a first field and a second field. The field value of the first field includes: the coordinates of multiple bone key points arranged in sequence; the field value of the second field includes: the connection information of each bone key point.

[0194] In a feasible implementation, when the processor 801 executes to display the bone image of the object according to the target bone information corresponding to the object, it is specifically used to: obtain the coordinates of each bone key point of the object and the connection information between each bone key point from the target array;

[0195] According to the coordinates of each bone key point of the object, the connection information between each bone key point, the color information of each bone key point, and the connection color information of each bone key point, the bone image of the object is displayed.

[0196] In a feasible implementation, before the processor 801 displays the skeletal image of the object based on the coordinates of each skeletal key point of the object, the connection information between each skeletal key point, the color information of each skeletal key point, and the connection color information of each skeletal key point, it is specifically used for: generating the field value of the third field in the target array according to the preset color information of each skeletal key point, and the field value of the third field includes: the color identifiers of each skeletal key point arranged in sequence; each field value in the third field corresponds one-to-one with each field value in the first field;

[0197] Generating the field value of the fourth field in the target array according to the preset color information of the connections between each skeletal key point, and the field value of the fourth field includes: the color identifiers of the connections between each skeletal key point arranged in sequence; each field value in the fourth field corresponds one-to-one with each field value in the second field;

[0198] Obtaining the color information of each skeletal key point according to the field value of the third field in the target array;

[0199] Obtaining the connection color information of each skeletal key point according to the field value of the fourth field in the target array.

[0200] In a feasible implementation, when the processor 801 executes to display the skeletal image of the object, it is specifically used for: displaying the skeletal image of the object in a transparent layer, and the transparent layer is located above the layer of the source image.

[0201] In a feasible implementation, when the processor 801 executes to obtain the adjustment instruction of the input skeletal image and generate the adjusted skeletal information of the object, it is specifically used for: receiving the adjustment operation of the specified skeletal key point of the input object through the graphic interface provided by the canvas plug-in, and obtaining the new coordinates of the specified skeletal key point;

[0202] According to the new coordinates of the specified skeletal key point, correcting the coordinates of the corresponding skeletal key point in the target skeletal information corresponding to the object, and generating the new skeletal information corresponding to the object.

[0203] In a feasible implementation, when the processor 801 executes to send the adjusted skeletal information to the server, it is specifically used for: sending the adjusted skeletal information to the server through the graphic interface provided by the canvas plug-in.

[0204] In a feasible implementation, when the processor 801 performs conversion processing on the source image of the user input to obtain the image data information of the source image and sends the image data information of the source image to the server, it is specifically used for: converting the data format of the source image to obtain the image data information of the source image; obtaining at least one preprocessing parameter information corresponding to the input source image; and sending the image data information of the source image and at least one preprocessing parameter information corresponding to the source image to the server.

[0205] In some embodiments, the processor 801 executes machine-readable instructions to perform the following steps:

[0206] Receiving the image information of the source image sent by the image recognition module of the client and parsing the image information of the source image to obtain the source image, where the source image includes at least one object;

[0207] Extracting the original bone information of the object from the source image;

[0208] Converting the original bone information of the object to obtain the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object;

[0209] Sending the target bone information corresponding to the object to the bone adjustment module of the client.

[0210] In a feasible implementation, when the processor 801 extracts the original bone information of the object from the source image, it is specifically used for: identifying the source image, determining the positions of each bone key point of the object in the source image and the connections between each bone key point, and taking the positions of each bone key point and the connections between each bone key point as the original bone information.

[0211] In a feasible implementation, when the processor 801 converts the original bone information of the object to obtain the target bone information corresponding to the object, it is specifically used for: determining the coordinates of each bone key point according to the positions of each bone key point;

[0212] Establishing the corresponding relationship between two bone key points connected to each other according to the connections between each bone key point;

[0213] Generating the field values of each field in the target array according to the coordinates of each bone key point and the corresponding relationship between two bone key points connected to each other, and taking the target array as the target bone information corresponding to the object.

[0214] In a feasible implementation, when the processor 801 generates the field values of each field in the target array according to the coordinates of each skeletal key point and the correspondence between two connected skeletal key points, it is specifically used to: generate the field value of the first field in the target array according to the coordinates of each skeletal key point, and the field value of the first field includes: the coordinates of multiple skeletal key points arranged in sequence.

[0215] Generate the field value of the second field in the target array according to the position of the coordinates of each skeletal key point in the field value of the first field and the correspondence between two connected skeletal key points.

[0216] In the above manner, the image recognition module and the skeleton adjustment module can be deployed on the client side of the electronic device at the same time, and the functions of the image recognition module and the skeleton adjustment module can be integrated. The image data information after the source image is processed by the image recognition module can be sent to the server for processing, and the target skeleton information corresponding to the object in the source image obtained after the server processes can be directly sent to the skeleton adjustment module, and the skeleton adjustment module can display the skeleton image of the object in the source image according to the target skeleton information. Compared with the prior art, the server does not need to send the target skeleton information corresponding to the object in the source image to the image recognition module, and then the image recognition module forwards the target skeleton information to an external skeleton adjustment module for processing, which simplifies the interaction process and improves the interaction efficiency. Moreover, through the operation dynamic response ability provided by the client, the adjustment operation of the skeleton image can be performed in the skeleton adjustment module without relying on a third-party plug-in, reducing the processing time consumed by the repeated transmission of data, thereby improving the efficiency of image processing.

[0217] Secondly, the server extracts the skeleton information of the object in the source image and converts the skeleton information of the object in the source image according to a preset format to obtain the target skeleton information in the preset format and send it back to the skeleton adjustment module of the client. Since the client and the server perform data interaction based on a unified data format, the skeleton adjustment module can quickly read the relevant information of the skeleton from the target skeleton information according to the preset format, so as to generate and display the skeleton image of the object. Performing data interaction based on a unified data format can improve the efficiency of data transmission and parsing processing.

[0218] Among them, the storage medium 802 stores program code, and when the program code is executed by the processor 801, the processor 801 is caused to execute various steps in the image editing processing method according to various exemplary embodiments of the present application described in the "Exemplary Method" part of this specification.

[0219] The processor 801 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0220] The storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory may include at least one type of storage medium, for example, it may include flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (RAM), a static random access memory (SRAM), a programmable read-only memory (PROM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, and so on. The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The storage medium 802 in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0221] Optionally, the embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the processor executes the following steps:

[0222] Perform conversion processing on the source image input by the user through an image recognition module to obtain the image data information of the source image, and send the image data information of the source image to the server, where the source image includes at least one object;

[0223] Receive the target bone information corresponding to the object in the source image sent by the server through the bone adjustment module, and display the bone image of the object according to the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object;

[0224] Obtain the adjustment instruction of the input bone image through the bone adjustment module, generate the adjusted bone information of the object; and send the adjusted bone information to the server.

[0225] In a feasible implementation, the target bone information includes a target array; the target array includes: a first field and a second field, and the field value of the first field includes: the coordinates of multiple bone key points arranged in sequence; the field value of the second field includes: the connection information of each bone key point.

[0226] In a feasible implementation, when the processor 801 executes to display the bone image of the object according to the target bone information corresponding to the object, it is specifically used for: obtaining the coordinates of each bone key point of the object and the connection information between each bone key point from the target array;

[0227] Display the bone image of the object according to the coordinates of each bone key point of the object, the connection information between each bone key point, the color information of each bone key point, and the connection color information of each bone key point.

[0228] In a feasible implementation, before the processor 801 executes to display the bone image of the object according to the coordinates of each bone key point of the object, the connection information between each bone key point, the color information of each bone key point, and the connection color information of each bone key point, it is specifically used for: generating the field value of the third field in the target array according to the preset color information of each bone key point, and the field value of the third field includes: the color identifiers of each bone key point arranged in sequence; each field value in the third field corresponds one-to-one with each field value in the first field;

[0229] Generate the field value of the fourth field in the target array according to the preset color information of the connection between each bone key point, and the field value of the fourth field includes: the color identifiers of the connections between each bone key point arranged in sequence; each field value in the fourth field corresponds one-to-one with each field value in the second field;

[0230] Obtain the color information of each bone key point according to the field value of the third field in the target array;

[0231] Obtain the connection color information of each bone key point according to the field value of the fourth field in the target array.

[0232] In a feasible implementation, when the processor 801 executes the skeletal image of the display object, it is specifically used to: display the skeletal image of the object in a transparent layer, and the transparent layer is located above the layer of the source image.

[0233] In a feasible implementation, when the processor 801 executes the adjustment instruction for obtaining the input skeletal image and generates the adjusted skeletal information of the object, it is specifically used to: receive the adjustment operation of the specified skeletal key points of the input object through the graphic interface provided by the canvas plug-in, and obtain the new coordinates of the specified skeletal key points;

[0234] According to the new coordinates of the specified skeletal key points, correct the coordinates of the corresponding skeletal key points in the target skeletal information corresponding to the object, and generate the new skeletal information corresponding to the object.

[0235] In a feasible implementation, when the processor 801 executes the sending of the adjusted skeletal information to the server, it is specifically used to: send the adjusted skeletal information to the server through the graphic interface provided by the canvas plug-in.

[0236] In a feasible implementation, when the processor 801 executes the conversion processing of the source image input by the user, obtains the image data information of the source image, and sends the image data information of the source image to the server, it is specifically used to: perform data format conversion on the source image to obtain the image data information of the source image; obtain at least one preprocessing parameter information corresponding to the input source image; send the image data information of the source image and at least one preprocessing parameter information corresponding to the source image to the server.

[0237] In some embodiments, the processor 801 executes machine-readable instructions to perform the following steps:

[0238] Receive the image information of the source image sent by the image recognition module of the client, and parse the image information of the source image to obtain the source image, and the source image includes at least one object;

[0239] Extract the original skeletal information of the object from the source image;

[0240] Convert the original skeletal information of the object to obtain the target skeletal information corresponding to the object; the target skeletal information includes the information of each skeletal key point of the object;

[0241] Send the target skeletal information corresponding to the object to the skeletal adjustment module of the client.

[0242] In a feasible implementation, when the processor 801 extracts the original skeleton information of an object from the source image, it is specifically used for: identifying the source image, determining the positions of the skeleton key points of the object in the source image and the connections between the skeleton key points, and taking the positions of the skeleton key points and the connections between the skeleton key points as the original skeleton information.

[0243] In a feasible implementation, when the processor 801 performs a conversion on the original skeleton information of the object to obtain the target skeleton information corresponding to the object, it is specifically used for: determining the coordinates of each skeleton key point according to the positions of the skeleton key points;

[0244] establishing a correspondence between two mutually connected skeleton key points according to the connections between the skeleton key points;

[0245] generating the field values of each field in the target array according to the coordinates of each skeleton key point and the correspondence between two mutually connected skeleton key points, and taking the target array as the target skeleton information corresponding to the object.

[0246] In a feasible implementation, when the processor 801 generates the field values of each field in the target array according to the coordinates of each skeleton key point and the correspondence between two mutually connected skeleton key points, it is specifically used for: generating the field value of the first field in the target array according to the coordinates of each skeleton key point, and the field value of the first field includes: the coordinates of multiple skeleton key points arranged in sequence;

[0247] generating the field value of the second field in the target array according to the position of the coordinates of each skeleton key point in the field value of the first field and the correspondence between two mutually connected skeleton key points.

[0248] In the above manner, an image recognition module and a skeleton adjustment module can be simultaneously deployed on the client side of the electronic device, the functions of the image recognition module and the skeleton adjustment module can be integrated, the image data information obtained after the source image is processed by the image recognition module can be sent to the server for processing, and the target skeleton information corresponding to the object in the source image obtained after the server processes can be directly sent to the skeleton adjustment module, and the skeleton adjustment module can display the skeleton image of the object in the source image according to the target skeleton information. Compared with the prior art, the server does not need to send the target skeleton information corresponding to the object in the source image to the image recognition module, and then the image recognition module forwards the target skeleton information to an external skeleton adjustment module for processing, which simplifies the interaction process and improves the interaction efficiency. Moreover, through the operation dynamic response ability provided by the client, adjustment operations on the skeleton image can be performed in the skeleton adjustment module without the need to execute with the help of a third-party plugin, reducing the processing time consumed by the repeated transmission of data, thereby improving the efficiency of image processing.

[0249] Secondly, the server extracts the skeletal information of the object in the source image and converts the skeletal information of the object in the source image into a preset format to obtain the target skeletal information in the preset format and sends it back to the skeletal adjustment module of the client. Since the client and the server perform data interaction based on a unified data format, the skeletal adjustment module can quickly read the relevant information of the skeleton from the target skeletal information according to the preset format, so as to generate and display the skeletal image of the object. Performing data interaction based on a unified data format can improve the efficiency of data transmission and parsing processing.

[0250] In the embodiment of the present application, when the computer program is run by the processor, it can also execute other machine-readable instructions to execute the methods described in other parts of the embodiment. For the specific method steps and principles of execution, refer to the description of the embodiment and will not be elaborated here.

[0251] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0252] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0253] In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of a combination of hardware and software functional units.

[0254] The integrated unit implemented in the form of software functional units can be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute some steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (English: Read-Only Memory, abbreviated as: ROM), random access memories (English: Random Access Memory, abbreviated as: RAM), magnetic disks, or optical discs that can store program codes.

Claims

1. An image editing and processing method, characterized in that, A client applied to an image editing system; an image recognition module and a skeleton adjustment module are deployed on the client, and the method includes: Performing conversion processing on a source image input by a user through the image recognition module to obtain image data information of the source image, and sending the image data information of the source image to a server, wherein at least one object is included in the source image; Receiving, by the skeleton adjustment module, target skeleton information corresponding to the object in the source image sent by the server, and displaying a skeleton image of the object according to the target skeleton information corresponding to the object; the target skeleton information includes information of each skeleton key point of the object; Obtaining, by the skeleton adjustment module, an adjustment instruction for the input skeleton image, generating adjusted skeleton information of the object; and sending the adjusted skeleton information to the server.

2. The method according to claim 1, characterized in that, The target skeleton information includes a target array, and the target array includes: a first field and a second field, and the field value of the first field includes: coordinates of a plurality of skeleton key points arranged in sequence; the field value of the second field includes: connection information of each skeleton key point.

3. The method according to claim 2, wherein The displaying the skeleton image of the object according to the target skeleton information corresponding to the object includes: Obtaining coordinates of each skeleton key point of the object and connection information between each skeleton key point from the target array; Displaying the skeleton image of the object according to the coordinates of each skeleton key point of the object, the connection information between each skeleton key point, the color information of each skeleton key point, and the connection color information of each skeleton key point.

4. The method according to claim 3, wherein Before the displaying the skeleton image of the object according to the coordinates of each skeleton key point of the object, the connection information between each skeleton key point, the color information of each skeleton key point, and the connection color information of each skeleton key point, it includes: Generating a field value of a third field in the target array according to preset color information of each skeleton key point, and the field value of the third field includes: color identifiers of each skeleton key point arranged in sequence; each field value in the third field corresponds one-to-one with each field value in the first field; Generating a field value of a fourth field in the target array according to preset color information of the connection between each skeleton key point, and the field value of the fourth field includes: color identifiers of the connection between each skeleton key point arranged in sequence; each field value in the fourth field corresponds one-to-one with each field value in the second field; Obtaining the color information of each skeleton key point according to the field value of the third field in the target array; Obtaining the connection color information of each skeleton key point according to the field value of the fourth field in the target array.

5. The method according to claim 1, characterized in that The displaying the skeleton image of the object includes: Displaying the skeleton image of the object in a transparent layer, and the transparent layer is located above the layer of the source image.

6. The method according to any one of claims 1-5, characterized in that The obtaining the adjustment instruction for the input skeleton image and generating the adjusted skeleton information of the object includes: Receiving, through a graphic interface provided by a canvas plugin, an adjustment instruction for a specified skeleton key point of the object in the input, and obtaining new coordinates of the specified skeleton key point; According to the new coordinates of the specified skeletal key points, correct the coordinates of the corresponding skeletal key points in the target skeletal information corresponding to the object to generate the new skeletal information corresponding to the object.

7. The method according to claim 1, characterized in that The sending the adjusted skeletal information to the server includes: Sending the adjusted skeletal information to the server through the graphic interface provided by the canvas plugin.

8. The method according to claim 1, wherein The converting and processing the source image input by the user to obtain the image data information of the source image and sending the image data information of the source image to the server includes: Converting the data format of the source image to obtain the image data information of the source image; Obtaining at least one piece of preprocessing parameter information corresponding to the input source image; Sending the image data information of the source image and at least one piece of preprocessing parameter information corresponding to the source image to the server.

9. An image editing and processing method, characterized in that, Applied to the server in the image editing system; The method includes: Receiving the image data information of the source image sent by the image recognition module of the client and parsing the image data information of the source image to obtain the source image, where the source image includes at least one object; Extracting the original skeletal information of the object from the source image; Converting the original skeletal information of the object to obtain the target skeletal information corresponding to the object; The target skeletal information includes the information of each skeletal key point of the object; Sending the target skeletal information corresponding to the object to the skeletal adjustment module of the client.

10. The method according to claim 9, wherein The extracting the original skeletal information of the object from the source image includes: Identifying the source image, determining the positions of each skeletal key point of the object in the source image and the connection lines between each skeletal key point, and taking the positions of each skeletal key point and the connection lines between each skeletal key point as the original skeletal information.

11. The method according to claim 10, characterized in that The converting the original skeletal information of the object to obtain the target skeletal information corresponding to the object includes: Determining the coordinates of each skeletal key point according to the positions of each skeletal key point; Establishing the corresponding relationship between two skeletal key points connected to each other according to the connection lines between each skeletal key point; Generating the field values of each field in the target array according to the coordinates of each skeletal key point and the corresponding relationship between two skeletal key points connected to each other, and taking the target array as the target skeletal information corresponding to the object.

12. The method according to claim 11, wherein The generating the field values of each field in the target array according to the coordinates of each skeletal key point and the corresponding relationship between two skeletal key points connected to each other includes: Generating the field value of the first field in the target array according to the coordinates of each skeletal key point, and the field value of the first field includes: the coordinates of multiple skeletal key points arranged in sequence; Generating the field value of the second field in the target array according to the position of the coordinates of each skeletal key point in the field value of the first field and the corresponding relationship between two skeletal key points connected to each other.

13. An image editing and processing device, characterized in that, Applied to the client in the image editing system; The image recognition module and the skeletal adjustment module are deployed on the client; The image recognition module is used to perform conversion processing on the source image input by the user, obtain the image information of the source image, and send the image data information of the source image to the server, where at least one object is included in the source image; The bone adjustment module is used to receive the target bone information corresponding to the object in the source image sent by the server, and display the bone image of the object according to the target bone information corresponding to the object; the target bone information is used to represent the information of each bone key point of the object; The bone adjustment module is used to obtain the adjustment instruction of the input bone image, generate the adjusted bone information of the object; and send the adjusted bone information to the server.

14. An image editing and processing device, characterized in that, Applied to a server in an image editing system; the device includes: a parsing module, a processing module, a conversion module, and a sending module; The parsing module is used to receive the image information of the source image sent by the image recognition module of the client, and parse the image information of the source image to obtain the source image, where at least one object is included in the source image; The processing module is used to extract the original bone information of the object from the source image; The conversion module is used to convert the original bone information of the object to obtain the target bone information corresponding to the object; the target bone information includes the information of each bone key point of the object; The sending module is used to send the target bone information corresponding to the object to the bone adjustment module of the client.

15. An electronic device, characterized in that, Comprising: A processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to perform the image editing processing method according to any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is run by the processor, it performs the image editing processing method according to any one of claims 1 to 12.

Images