Image processing method, system, online dressing system, electronic device

Abstract

Embodiments of the present application provide an image processing method, system, online dressing system, and electronic device. Among them, the image processing method includes: obtaining an original image to be subject to matte extraction processing and displaying the original image; in response to a matte extraction operation on the original image, sending the original image to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to image content with scene semantics; determining a matte extraction region based on the position information and a selection operation on the candidate matte extraction regions in the original image; and performing matte extraction processing on the original image based on the position information of the matte extraction region. This method analyzes the original image using a target granularity to obtain candidate matte extraction regions with scene semantics, which helps to reduce the occurrence of missed matte extraction and over-matte extraction when selecting a matte extraction region and improves the matte extraction quality.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to an image processing method, an image processing system, an online dressing system, an electronic device, a storage medium, and a computer program product. Background Art

[0002] Matting refers to extracting the image of an object from a picture. Matting has a wide range of applications in the e-commerce scenario. For example, after extracting the images of objects such as clothing and models from the original picture, model try-on images can be regenerated based on the extracted object images. However, in the e-commerce scenario, there are complex scenarios with multiple objects in the pictures. The existing matting methods often have the following defects in the application process in the e-commerce scenario: the matting quality is poor, and there are cases of over-matting or missing matting.

[0003] It can be seen that the existing image processing methods still need to be improved. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method, which can improve the matting quality and reduce the occurrence of over-matting and missing matting.

[0005] Correspondingly, the embodiments of the present application also provide an image processing system, an online dressing system, an electronic device, a storage medium, and a computer program product to ensure the implementation and application of the above image processing method.

[0006] To solve the above problems, the embodiments of the present application disclose an image processing method, which is applied to a client. The method includes:

[0007] Obtain an original picture to be processed by matting, and display the original picture;

[0008] In response to a matting operation on the original picture, send the original picture to a preset server, so that the preset server analyzes and processes the original picture based on a target granularity to obtain candidate matting regions in the original picture and position information of the candidate matting regions, where the target granularity corresponds to picture content with scene semantics;

[0009] Based on the position information and a selection operation on the candidate matting regions in the original picture, determine the matting regions;

[0010] Based on the position information of the matting regions, perform matting processing on the original picture.

[0011] The embodiments of the present application also disclose an image processing method, which is applied to a server. The method includes:

[0012] In response to receiving the original image to be processed for matte extraction sent by the client, analyze and process the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics;

[0013] Send the position information to the client, so that the client determines the matte extraction region based on the position information and the selection operation on the candidate matte extraction regions in the original image, and sends the matte data of the matte extraction region to the server;

[0014] In response to receiving the matte data sent by the client, perform matte extraction processing on the original image.

[0015] An embodiment of the present application discloses an image processing method, including:

[0016] Obtain the original image to be processed for matte extraction and display the original image;

[0017] In response to the matte extraction operation on the original image, analyze and process the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics;

[0018] Determine the matte extraction region based on the position information and the selection operation on the candidate matte extraction regions in the original image;

[0019] Obtain the matte data of the matte extraction region based on the position information of the matte extraction region;

[0020] Perform matte extraction processing on the original image based on the matte data.

[0021] An embodiment of the present application discloses an image processing system, the object generation system includes: a client and a server, where,

[0022] The client is used to execute the image processing method applied to the client in the embodiment of the present application;

[0023] The server is used to execute the image processing method applied to the server in the embodiment of the present application.

[0024] An embodiment of the present application discloses an online dressing system, the object generation system includes: a client and a server, where,

[0025] The client is used to obtain the original image and display the original image;

[0026] The client is further configured to obtain dressing configuration information based on a user's selection operation, where the dressing configuration information includes one or more of the following: model information, reserved clothing types, and generated image configuration information;

[0027] The client is further configured to send the original image to a preset server in response to an editing operation on the original image;

[0028] The server is configured to analyze and process the original image based on a target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions, and send the position information to the client, where the target granularity corresponds to image content with scene semantics;

[0029] The client is further configured to determine a matte region based on the position information and a selection operation on the candidate matte region in the original image;

[0030] The client is further configured to send matte data of the matte region and the dressing configuration information to the server;

[0031] The server is further configured to generate a model dressing image based on the original image, the matte data, and the dressing configuration information, and send the model dressing image to the client;

[0032] The client is further configured to display the model dressing image.

[0033] An embodiment of the present application also discloses a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method as described in the embodiment of the present application.

[0034] An embodiment of the present application also discloses a computer program product, including a computer program / computer-executable instructions, characterized in that the computer program / computer-executable instructions, when executed by a processor in an electronic device, implement the method as described in the embodiment of the present application.

[0035] Compared with the prior art, the embodiments of the present application have the following advantages:

[0036] By obtaining the original image to be processed for matte extraction and displaying the original image, and then, in response to the matte extraction operation on the original image, sending the original image to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and the position information of the candidate matte regions, where the target granularity corresponds to the image content with scene semantics, so that the matte region determined based on the position information and the selection operation of the candidate matte regions in the original image better meets the user's matte extraction requirements, helps to improve the matte extraction quality, and reduces the occurrence of over-matte extraction and under-matte extraction. On the other hand, when performing matte extraction processing on the original image based on the position information of the matte region, the interaction times for determining the matte region can be effectively reduced, and the matte extraction efficiency can be improved. Description of the Drawings

[0037] Figure 1 is one of the flowcharts of the steps of the image processing method disclosed in the embodiments of the present application;

[0038] Figure 2 is a schematic diagram of the original image in the image processing method disclosed in the embodiments of the present application;

[0039] Figure 3 is the analysis and processing of the original image shown in Figure 2 to obtain a schematic diagram of the candidate matte regions;

[0040] Figure 4 is a schematic diagram of the effect of vector-drawing candidate matte regions based on the edge contour before optimization in the image processing method disclosed in the embodiments of the present application;

[0041] Figure 5 Schematic diagram of the effect of vector-drawing candidate matte regions based on the optimized edge contour in the image processing method disclosed in the embodiments of the present application;

[0042] Figure 6 is the analysis and processing of the original image shown in Figure 2 to obtain a schematic diagram of the result of the region selection operation of the candidate matte regions in the original image;

[0043] Figure 7 is another flowchart of the steps of the image processing method disclosed in the embodiments of the present application;

[0044] Figure 8 is a schematic diagram of the dual drawing board interface displayed by the client in the image processing method disclosed in the embodiments of the present application;

[0045] Figure 9 is a schematic diagram of the effect of matte region selection in the image processing method disclosed in the embodiments of the present application;

[0046] Figure 10 It is the third step flowchart of the image processing method disclosed in the embodiments of the present application;

[0047] Figure 11 It is the fourth step flowchart of the image processing method disclosed in the embodiments of the present application;

[0048] Figure 12 It is the working flowchart of the online dressing system disclosed in the embodiments of the present application;

[0049] Figure 13 It is the structural schematic diagram of an exemplary device provided by an embodiment of the present application. Detailed implementation manners

[0050] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0051] The image processing method disclosed in the embodiments of the present application can be applied to applications such as model skin replacement, intelligent dressing, and intelligent background switching. When the existing matte extraction algorithm performs image extraction of the target subject, due to the defects of the segmentation algorithm, problems such as over-extraction and missed extraction, that is, inaccurate matte extraction, often occur. On the other hand, in some existing image processing methods, when the user selects certain areas, the selected areas are presented in a specified color. Due to the slow response speed of image segmentation, there is no obvious visual change in the interface when repeatedly selecting or deselecting areas, resulting in a poor user experience.

[0052] The image processing method disclosed in the embodiments of the present application divides the image into the smallest granularity with scene semantics, and improves the human-computer interaction solution in the matte extraction stage to reduce the occurrence of problems such as missed extraction and over-extraction. On the other hand, by making full use of the user's information input to determine the final matte extraction area, it plays a decisive role in accurately and quickly determining the matte extraction area and improving the matte extraction quality.

[0053] Refer to Figure 1 The image processing method disclosed in the embodiments of the present application is applied to a client, and the method includes: Step 102 to Step 108.

[0054] Step 102, obtain the original image to be processed for matte extraction, and display the original image.

[0055] In some alternative embodiments, an image upload entry may be set on the client, through which a user can upload the original pictures stored locally on the client. In some other alternative embodiments, a picture browsing entry may also be set on the client, through which a user can access network pictures (such as pictures of virtual model clothing try-on on an e-commerce platform) and select the pictures that need to be subject to matte processing. In the embodiments of the present application, the specific implementation manners of obtaining the original pictures to be subject to matte processing are not limited.

[0056] Further, the client can create a drawing board for drawing the obtained original pictures.

[0057] Step 104: In response to a matte operation on the original picture, send the original picture to a preset server, so that the preset server analyzes and processes the original picture based on a target granularity to obtain candidate matte regions in the original picture and position information of the candidate matte regions, where the target granularity corresponds to picture content with scene semantics.

[0058] Optionally, the scene semantics refer to the meaning of physical objects in a certain scene, and the picture content with scene semantics refers to picture content with specific meanings in the scene, including a number of pixel points. For example, for a virtual model picture in an intelligent clothing try-on scene, the picture content with scene semantics includes, but is not limited to: hats, faces, collars, hair, arms, upper clothes, skirts, background blocks, etc. Compared with the picture content with scene semantics, each pixel point in the virtual model picture can only indicate whether the position is the foreground or the background, and cannot accurately indicate which part of the virtual model the position constitutes. Therefore, in the prior art, when performing image segmentation based on pixel points, situations of missed segmentation and over-segmentation are likely to occur. For example, multiple virtual models in a picture are segmented as one subject, or the prosthetic limb of a virtual model is missed in segmentation, resulting in problems of missed matte and over-matte when performing matte based on the segmentation result.

[0059] The target granularity described in the embodiments of the present application is determined according to the smallest component with scene semantics of the target subject. The target granularity is obtained by a preset segmentation model through learning from training data. For example, for a picture of a virtual model or a real model trying on clothes, the target granularity may be: each part of the clothing (such as collars, sleeves, the main body, trousers, etc.) and each part of the human body (such as hair, face, neck, arms, hands, etc.). Another example is that when a preset segmentation model is trained based on plant pictures, the target granularity may be flowerpots, flowers, branches, fruits, etc.

[0060] In some alternative embodiments, the preset segmentation model is obtained by optimizing the parameters of the SAM (Segment Anything Model) model with the goal of segmenting the physical components of the target object. SAM is called a pre-trained generative model in the field of computer vision. Different from the traditional simple semantic segmentation method, SAM incorporates a Prompt mechanism, which can use text, coordinate points, coordinate frames, etc. as auxiliary information to optimize the segmentation results. At the same time, due to its massive training dataset and model structure design, SAM can achieve zero-shot segmentation.

[0061] However, the inventors found during the segmentation test of the open-source image set in the prior art using the pre-trained SAM that, due to the too fine-grained automated segmentation, when performing matte extraction based on the segmentation results output by SAM, the obtained parts do not have scene semantics. For example, for a mannequin clothing try-on image, each small joint point will be segmented, and it is necessary to select multiple times to select the image area that needs to be matte-extracted, increasing the number and difficulty of user interactions and affecting the matte extraction experience.

[0062] In view of this, the inventors conducted multiple rounds of debugging on the SAM model parameters by adjusting the Prompt (prompt words). For example, after optimizing the training through hyperparameter tuning such as controlling the density of sampling points, setting the intersection over union threshold, and setting the minimum area of the returned mask, etc., a parameter combination relatively suitable for the target scenario (such as the intelligent dressing scenario) was found, and the SAM after parameter adjustment was used as the preset segmentation model.

[0063] Optionally, for the specific implementation of the multiple rounds of debugging of the SAM model parameters, refer to the prior art, which will not be elaborated in the embodiments of the present application.

[0064] In the matte extraction stage, after inputting the original image of the image to be matte-extracted into the preset segmentation model, the preset segmentation model performs segmentation processing on the original image to obtain a pixel point matrix of the segmentation region. Among them, the segmentation region is automatically segmented by the preset segmentation model, and after hyperparameter tuning such as controlling the density of sampling points, setting the intersection over union threshold, and setting the minimum area of the returned mask, it is the region where the image content with scene semantics generated by matte extraction is located. Taking Figure 2 the original image shown as an example, the preset segmentation model is called to perform analysis and processing on the original image based on the target granularity, and the candidate matte extraction regions in the original image can include, for example Figure 3 those shown in: the chest, neck, hat, skirt, etc. of the clothes in the original image. Among them, the pixel point matrix refers to a two-dimensional array corresponding to the candidate matte extraction region generated by segmenting the original image by the preset segmentation model after hyperparameter tuning, and each element represents a pixel point in the original image. In the pixel point matrix, the horizontal index represents the width of the original image, and the vertical index represents the height of the original image.

[0065] Optionally, the pixel point matrix can be used as the position information of the candidate matte extraction region for subsequent matte extraction processing.

[0066] by Figure 3 It can be seen that the candidate matte extraction regions corresponding to the pixel point matrix output by the preset segmentation model are the chest, neck, hat, etc. of the clothes in the original picture, which have scene semantics and are closer to the matte extraction regions that the user may select. This can reduce the number of times of region selection on the original picture when the user selects the matte extraction region, reduce the matte extraction interaction cost, and effectively reduce the phenomena of missed matte extraction and over-matte extraction.

[0067] Furthermore, the application layer performs image segmentation processing according to the position represented by the pixel point matrix of the candidate matte extraction region, and then the image region corresponding to the corresponding candidate matte extraction region can be extracted. However, the pixel point matrix stores the information of all pixels on the picture, and the data volume is relatively large, reaching the MB level. During the matte extraction process, due to the large amount of calculation, there is a certain delay in the front-end user interaction, resulting in a poor user interaction experience. However, in the matte extraction scenario, only by knowing the horizontal / vertical index positions of the edge pixels of the segmented matte extraction region in the whole picture can the purpose of accurate matte extraction be achieved. In some embodiments of the present application, in order to reduce the amount of data processed for image segmentation, the pixel point matrix is further optimized to generate a vector for drawing the edge contour of the selection area, so that the data volume returned to the front-end application is reduced from the MB (megabyte) level to the KB (kilobyte) level. In this way, when the front-end application fills colors on the original picture or draws the matte extraction region, it can change from drawing all pixel points to drawing the region based on the edge contour, greatly improving the human-computer interaction speed of matte extraction.

[0068] In some optional embodiments, sending the original picture to a preset server, so that the preset server analyzes and processes the original picture based on a target granularity to obtain the candidate matte extraction region and the position information of the candidate matte extraction region in the original picture includes: sending the original picture to the preset server, so that the preset server analyzes and processes the original picture based on a target granularity to obtain the candidate matte extraction region and the pixel point matrix of the candidate matte extraction region in the original picture; performing region contour detection based on the pixel point matrix to obtain a vector of the edge contour in the original picture; performing region splitting processing on the vector to obtain a vector for drawing the edge contour of each candidate matte extraction region as the position information of the corresponding candidate matte extraction region.

[0069] Call the preset segmentation model to perform target-granularity analysis and processing on the original image, and obtain the candidate matte regions in the original image. For the specific implementation of the pixel point matrix of the candidate matte regions, refer to the prior art and will not be elaborated in the embodiments of this application.

[0070] Next, based on the findContours method in the OpenCV library, detect the region contours in the pixel point matrix. The region contours are curves formed by points continuous along the boundary of the region in the image.

[0071] The CV2.findContours() method is used for contour detection. After taking the original image as the input image and inputting it into the findContours() method function, the findContours() method function will return a list, which is a two-dimensional matrix, and each element in the list is a contour in the input image. In the embodiments of this application, in combination with the actual scenario, the mode of only detecting the outermost contour is used, such as the CV2.RETR_EXTERNAL mode, to detect the external contour of the input image and obtain the output result. After compression, the output result is a two-dimensional matrix, that is, a pixel point matrix. When the client redraws the mask layer based on the edge contour points, each vector in this two-dimensional matrix will be traversed, which will cause the vectors corresponding to different blocks to be connected together, resulting in an error in drawing the mask layer. That is, since there are multiple candidate matte regions corresponding in the pixel point matrix, after detecting the contours, a new two-dimensional array composed of edge contour points will be generated. If the contour is directly drawn based on the newly generated two-dimensional array, since there are multiple contour vectors in this two-dimensional array, there will be a line connecting the heads and tails between the originally unconnected candidate matte regions, as Figure 4 shown.

[0072] In the embodiments of this application, the server adds a dimension to each vector in the two-dimensional matrix (i.e., the pixel point matrix) returned by the CV2.findContours() function and wraps it into a two-dimensional matrix. For example, the one-dimensional vector [1, 2, 3] is wrapped into a two-dimensional vector [[1, 2, 3]]. Thus, the two-dimensional matrix of edge contour points becomes a three-dimensional matrix, which can realize the drawing of each region. In this way, when the front-end application depicts the edge contours of the candidate matte regions, the situation where the heads and tails of multiple candidate matte regions are connected will not occur. The schematic diagram of the candidate matte region drawn based on the vector of the edge contour obtained after splitting is shown in Figure 5 shown.

[0073] Optionally, the server adopts a contour approximation method, selects the CV2.CHAIN_APPROX_SIMPLE mode, compresses the data points in the horizontal and vertical directions, improves the transmission efficiency of data between the client and the server, and improves the interaction speed.

[0074] In some embodiments of the present application, the vectors in the two-dimensional array regenerated by disassembling are multiple two-dimensional arrays, such that each candidate matte region corresponds to a two-dimensional data, that is, the vector of the edge contour of each candidate matte region is disassembled, and each is passed to the front-end application for drawing the matte of the candidate matte region, which can greatly shorten the matte drawing time and improve the human-computer interaction speed.

[0075] In some embodiments of the present application, other methods may also be used to perform regional splitting processing on the vector to obtain the vectors for drawing the edge contours of the candidate matte regions, which will not be listed one by one here. The specific implementation manner of performing regional splitting processing on the vector to obtain the vectors for drawing the edge contours of the candidate matte regions in the embodiments of the present application is not limited.

[0076] The image processing method disclosed in some embodiments of the present application can be implemented in an image processing system. For example, a human-computer interaction interface with the user is set on the client side. The user uploads or selects an original image to be matted through the human-computer interaction interface of the client side. Then, in response to the user triggering a matte operation on the original image, the client side sends the original image to the server side of the image processing system. The server side calls a preset segmentation model to perform object analysis processing on the original image based on the target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions. Then, the server side returns the obtained position information to the client side, and the client side displays the candidate matte regions.

[0077] Step 106, determine the matte region based on the position information and the selection operation on the candidate matte regions in the original image.

[0078] Next, the client side displays the candidate matte regions selected by the user according to the selection operation of the user on the candidate matte regions in the original image on the client side interface, and after the user confirms the matte, the matte region is obtained by integrating the candidate matte regions selected by the user.

[0079] In some optional embodiments, the determining the matte region based on the position information and the selection operation on the candidate matte regions in the original image includes: in response to detecting a region selection operation on the candidate matte regions in the original image, determining a first target candidate matte region targeted by the region selection operation according to the image position corresponding to the region selection operation and the position information of the candidate matte regions; updating the current selection state of the first target candidate matte region; performing color filling processing on the first target candidate matte region based on the updated selection state; and updating the matte region based on the updated selection state of the first target candidate matte region.

[0080] Among them, the area selection operation includes, but is not limited to: double-click operation, single-click operation, right-click selection operation, touch point lift operation, etc.

[0081] Determining the first target candidate matte region targeted by the area selection operation according to the position of the picture corresponding to the area selection operation and the position information of the candidate matte region includes: obtaining the position in the original picture corresponding to the area selection operation as the picture position; determining the candidate matte region where the picture position is located according to the position inclusion relationship between the candidate matte region determined according to the position information and the picture position, as the candidate matte region targeted by the area selection operation, denoted as the "first target candidate matte region". That is, according to the position coordinates of the area selection operation, it is determined which candidate matte region the area selection operation falls into.

[0082] For example, the user can click on any position in the original picture displayed on the client interface. After the client detects the click operation on the original picture, it first determines the position of the click operation in the original picture, denoted as the "picture position". Then, the client further determines which candidate matte region the current click operation falls into according to the position information of each candidate matte region obtained in the previous step, and takes the candidate matte region where the current click operation falls as the first target candidate matte region.

[0083] In some optional embodiments, the selection state includes: a selected state indicating that the first target candidate matte region has been selected as a matte region, and an unselected state indicating that the first target candidate matte region has not been selected as a matte region. Updating the current selection state of the first target candidate matte region includes: in response to the current selection state of the first target candidate matte region being the selected state, updating the selection state to the unselected state; in response to the current selection state of the first target candidate matte region being the unselected state, updating the selection state to the selected state.

[0084] In some optional embodiments, the user can add the selected candidate matte region as a matte region by performing a selection operation (such as a single-click operation) on a candidate matte region in the original picture that is in the unselected state. On the other hand, the user can also perform a selection operation (such as a single-click operation) on a candidate matte region in the selected state to cancel the selected state of a candidate matte region, that is, remove a candidate matte region from the matte regions.

[0085] In the embodiments of the present application, in the initial state, the client can initialize the selection status of each candidate matte region obtained in the foregoing steps to the "unselected state". After that, when the user selects a certain candidate matte region (i.e., the foregoing first target candidate matte region) as the matte region, the client adds the first target candidate matte region to the matte region and sets the selection status of the first target candidate matte region to the "selected state". When the user selects a candidate matte region that has already been selected as the matte region (i.e., the foregoing first target candidate matte region), the client removes the first target candidate matte region from the matte region and sets the selection status of the first target candidate matte region to the "unselected state".

[0086] After updating the selection status of the candidate matte region according to the user's region selection operation, in the embodiments of the present application, further perform color filling processing on the first target candidate matte region based on the updated selection status, so as to make an intuitive response to the user's region selection operation, return an operation result to the user, and improve the user interaction experience.

[0087] In some optional embodiments, the selection status includes: a selected state indicating that the first target candidate matte region has been selected as the matte region, and an unselected state indicating that the first target candidate matte region has not been selected as the matte region. The performing color filling processing on the first target candidate matte region based on the updated selection status includes: in response to the updated selection status of the first target candidate matte region being the selected state, filling the first target candidate matte region with a first preset color; in response to the updated selection status of the first target candidate matte region being the unselected state, clearing the first preset color filled in the first target candidate matte region.

[0088] In the embodiments of the present application, by filling the specified color in the matte region selected by the user, the matte region that has been selected is displayed to the user, and the display effect is as Figure 6 shown. For example, when the updated selection status of the first target candidate matte region selected by the user is the selected state, fill the first target candidate matte region with purple. When the updated selection status of the first target candidate matte region selected by the user is the unselected state (such as when the user cancels using the first target candidate matte region as the matte region), clear the color filled in the first target candidate matte region and restore the content in the original picture displayed in the first target candidate matte region. Optionally, the first target candidate matte region can be filled with the first preset color by adding a first preset color mask to the first target candidate matte region. Correspondingly, by canceling the mask of the first target candidate matte region, the color filled in the first target candidate matte region can be cleared.

[0089] Finally, update the matte region according to the updated selection status of the first target candidate matte region.

[0090] In some alternative embodiments, the selection status includes: a selected status indicating that the first target candidate matte region has been selected as the matte region, and an unselected status indicating that the first target candidate matte region has not been selected as the matte region. Updating the matte region based on the updated selection status of the first target candidate matte region includes: in response to the updated selection status of the first target candidate matte region being the selected status, adding the first target candidate matte region as the matte region; in response to the updated selection status of the first target candidate matte region being the unselected status, removing the first target candidate matte region from the matte region.

[0091] Optionally, the client can establish a set of matte regions. After each time the user performs a region selection operation, further update the matte regions included in the set according to the updated selection status of the first target candidate matte region corresponding to the region selection operation.

[0092] In some embodiments of the present application, in order to clearly show the selected matte regions to the user, two drawing boards are set on the client. Among them, one drawing board is used to display the original image, and the other drawing board is used to display the preview image of the matte regions selected by the user. As Figure 7 shown, after sending the original image to a preset server, such that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions, the method further includes: step 105.

[0093] Step 105, on the interface for displaying the original image, display a preview drawing board of the matte region in comparison.

[0094] As Figure 8 shown, on the client interface, there is a drawing board 802 for displaying the original image and a preview drawing board 804 for displaying the preview image of the matte region.

[0095] Correspondingly, after determining the matte region based on the position information and the selection operation on the candidate matte regions in the original image, the method further includes: step 107.

[0096] Step 107, based on the position information, draw the matte region in real time in the preview drawing board.

[0097] Optionally, when the user does not select any candidate matte region, the preview drawing board 804 displays a blank board. After the user selects a certain candidate matte region of the original image displayed in the drawing board 802, such as when the user selects the hat of the mannequin, the client will display in real time on the preview drawing board 804 a mask image of the specified color corresponding to the hat of the mannequin to prompt the user of the currently selected matte region. For example, if the user selects the hat, top, and skirt of the mannequin as the matte regions after performing region selection operations on each region of the original image displayed in the drawing board 802, the content displayed in the preview drawing board 804 includes the mask images of the hat, top, and skirt in the original image, and the display effect is as Figure 9 shown.

[0098] In the embodiments of the present application, by setting a dual drawing board on the client side to display in real time the matte regions selected by the user, it helps the user to confirm the matte regions in real time, improves the selection efficiency of the matte regions, and thus improves the matte extraction efficiency.

[0099] In the embodiments of the present application, based on the position information of the candidate matte regions determined in the foregoing steps, through position mapping, the position information of the corresponding matte regions in the preview drawing board 804 is obtained. Then, based on the position information of the matte regions, mask images of each matte region are drawn in real time in the preview drawing board 804. Among them, for the specific implementation of obtaining the position information of the corresponding matte regions in the preview drawing board 804 through position mapping based on the position information of the candidate matte regions, reference can be made to the prior art and will not be elaborated here. As described above, the position information of the candidate matte regions is a vector of the optimized edge contour. Therefore, the position information of the matte regions obtained after mapping is also a vector of the edge contour. In the embodiments of the present application, the mask images of the picture regions are drawn based on the vectors of the edge contours of the picture regions. Compared with drawing the mask images based on the pixel point matrix, the drawing speed is significantly improved, and the effect of real-time display can be achieved.

[0100] In some other optional embodiments, after responding to the matte extraction operation on the original image, sending the original image to a preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions, as Figure 10 shown, the method further includes: step 1041 and step 1042.

[0101] Step 1041, in response to detecting a region preview operation on the candidate matte region in the original image, determine a second target candidate matte region targeted by the region preview operation according to the picture position corresponding to the region preview operation and the position information of the candidate matte region.

[0102] Among them, the area preview operation includes, but is not limited to, mouse hovering operation, touch sliding operation, etc.

[0103] Determining a second target candidate matte region targeted by the area preview operation according to the picture position corresponding to the area preview operation and the position information of the candidate matte regions includes: obtaining the position in the original picture corresponding to the area preview operation as the picture position; determining the candidate matte region where the picture position is located according to the position inclusion relationship between the candidate matte region determined according to the position information and the picture position, as the candidate matte region targeted by the area preview operation, denoted as the "second target candidate matte region". That is, according to the position coordinates of the area preview operation, it is determined which candidate matte region the area preview operation falls into.

[0104] In the specific implementation process, when the client detects that the mouse hovers over the original picture, by obtaining the position coordinates carried by the mouse hovering event, the picture position of the mouse hovering position in the original picture can be calculated. Then, based on the position information of each candidate matte region of the original picture, the position inclusion relationship between the picture position and each candidate matte region is calculated respectively, and the candidate matte region containing the picture position is determined, and the candidate matte region containing the picture position is denoted as the second target candidate matte region. Among them, the specific implementation manner of determining the inclusion relationship between the picture area and the picture position refers to the prior art and will not be elaborated here.

[0105] For the specific implementation manner of determining the second target candidate matte region targeted by the area preview operation according to the picture position corresponding to the area preview operation and the position information of the candidate matte regions, refer to the specific implementation manner of determining the second target candidate matte region targeted by the area selection operation in the foregoing embodiments, which will not be elaborated here.

[0106] Step 1042, based on the current selection state of the second target candidate matte region, perform real-time highlighting display on the second target candidate matte region with a corresponding preset color.

[0107] Next, the client further highlights the second target candidate matte region to prompt the candidate matte region corresponding to the mouse or touch point of the user. In the embodiments of the present application, in order to further prompt whether the candidate matte region corresponding to the current mouse or touch point has been selected as the matte region, candidate matte regions in different selection states are highlighted with different colors.

[0108] In some alternative embodiments, the selection state includes: a selected state indicating that the second target candidate matte region has been selected as a matte region, and an unselected state indicating that the second target candidate matte region has not been selected as a matte region. Based on the current selection state of the second target candidate matte region, performing real-time highlighting of the second target candidate matte region in a corresponding preset color includes: in response to the current selection state of the second target candidate matte region being the selected state, performing real-time highlighting of the second target candidate matte region in a second preset color; and in response to the current selection state of the second target candidate matte region being the unselected state, performing real-time highlighting of the second target candidate matte region in a third preset color.

[0109] Optionally, performing real-time highlighting of the second target candidate matte region in a second preset color includes: adding a mask of the second preset color to the second target candidate matte region in real time; and performing real-time highlighting of the second target candidate matte region in a third preset color includes: adding a mask of the third preset color to the second target candidate matte region in real time.

[0110] Taking Figure 2 the original picture shown in

[0111] as an example, when the hat and upper garment of the mannequin have been selected as matte regions and the skirt has not been selected as a matte region, when the mouse hovers over the candidate matte region corresponding to the hat, the candidate matte region where the hat is located can be filled with orange or an orange mask can be added to prompt the user that this candidate matte region has been selected as a matte region. When the mouse hovers over the candidate matte region corresponding to the skirt, the candidate matte region corresponding to the skirt can be filled with lavender or a lavender mask can be added to prompt the user that this candidate matte region has not been selected as a matte region.

[0112] In some alternative embodiments, after determining the matte region based on the position information and the selection operation of the candidate matte region in the original image, the following steps are further included: in response to detecting an operation to clear the matte region with one key, clear the matte region; based on the position information, real-time clear the filled color in the original image and the drawn matte region in the preview drawing board. For example, a button indicating to clear the selected matte region with one key can be set on the interface where the original image and the preview of the matte region are displayed on the client side. By triggering this button, the user can clear all the selected matte regions with one key.

[0113] In some other alternative embodiments, after determining the matte region based on the position information and the selection operation of the candidate matte region in the original image, the following steps are further included: in response to detecting an operation to roll back to the previous matte region selection, roll back to the previously selected matte region; update the current selection state of the candidate matte region according to the previously selected matte region; perform real-time color filling processing on the candidate matte regions in the original image based on the updated selection state and the position information, and update the drawn matte region in the preview drawing board based on the updated selection state. For example, a button indicating to roll back to the previous operation can be set on the interface where the original image and the preview of the matte region are displayed on the client side. By triggering this button, the user can roll back the selection state of the candidate matte region in sequence. The client will refresh the selected candidate matte regions in the original image and the drawn matte region in the preview image in real time according to the rolled-back selection state of the candidate matte region.

[0114] In the embodiments of the present application, a way of adding a mask to a specified region of the original image can be adopted to prominently display the candidate matte region currently previewed or selected by the user. For example, add a specified color mask in base64 format to the first target candidate matte region or the second target candidate matte region.

[0115] Step 108: Send the position information of the matte region to the preset server, triggering the preset server to perform matte processing on the original image based on the position information.

[0116] In some alternative embodiments, optionally, based on the position information of the matte region, performing matte processing on the original image includes: sending the matte data corresponding to the matte region to the preset server, so that the preset server performs matte processing on the original image based on the matte data to obtain the matte region image. For example, after the client detects that the user confirms the matte region, it sends the matte data corresponding to the matte region in the original image in the preview drawing board to the server. After receiving the matte data, the server analyzes and processes the original image based on the matte data to obtain the image content of the region covered by the matte in the original image. Taking Figure 9 the matte region shown as an example, the server will extract the images of the hat, upper garment, and skirt in the mannequin picture from the original image.

[0117] For the specific implementation manner in which the server performs matte processing on the original image according to the matte data, refer to the prior art and will not be elaborated here.

[0118] In summary, for the image processing method disclosed in the embodiments of the present application, by obtaining the original image to be matte processed and displaying the original image, and then, in response to a matte operation on the original image, sending the original image to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions, where the target granularity corresponds to the image content with scene semantics, so that based on the position information and the matte region determined by the selection operation on the candidate matte regions in the original image, it is more in line with the user's matte requirements, helps to improve the matte quality, and reduces the occurrence of over-matting and under-matting situations. On the other hand, when performing matte processing on the original image based on the position information of the matte region, the interaction times for determining the matte region can be effectively reduced, and the matte efficiency can be improved.

[0119] Based on the above embodiments, the embodiments of the present application also disclose an image processing method, which is applied to a server. As Figure 11 shown, the image processing method includes: Step 1102 to Step 1106.

[0120] Step 1102, in response to receiving the original image to be matte processed sent by the client, analyzing and processing the original image based on a target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions, where the target granularity corresponds to the image content with scene semantics.

[0121] For the method of obtaining the original image to be matte processed, refer to the relevant descriptions in the foregoing embodiments and will not be elaborated here.

[0122] The specific implementation of the server analyzing and processing the original image based on the target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions can be found in the relevant descriptions in the previous embodiments, and will not be elaborated here.

[0123] Preferably, the position information is the vector of the edge contour.

[0124] Step 1104: Send the position information to the client, so that the client determines the matte region based on the position information and the selection operation on the candidate matte region in the original image, and sends the matte data of the matte region to the server.

[0125] After the server obtains the position information of each candidate matte region in the original image by calling a preset segmentation model, it sends the position information to the client. The client can further determine each candidate matte region in the original image based on the position information, and interact with the user in response to the user's preview operation or selection operation on the region in the original image, so as to determine the matte region according to the user's selection.

[0126] The specific implementation of the client determining the matte region based on the position information and the selection operation on the candidate matte region in the original image can be found in the previous description, and will not be elaborated here.

[0127] As described above, during the interaction process in which the client responds to the user's selection of the candidate matte region, for the candidate matte region selected by the user as the matte region, a matte will be added to the candidate matte region in the original image to prompt the user of the selection result of the matte region. After the user confirms the selected matte region, the client further sends the matte data added to the original image, that is, the matte data of the matte region, to the server. The matte data represents the region to be cut out in the original image.

[0128] Step 1106: In response to receiving the matte data sent by the client, perform matte processing on the original image.

[0129] After the server receives the matte data sent by the client, it performs matte processing on the original image based on the matte data to obtain the image of the matte region.

[0130] The specific implementation of performing matte processing on the original image based on the regional matte data can be found in the prior art, and will not be elaborated here.

[0131] In summary, the image processing method disclosed in the embodiments of the present application, by responding to the received original image to be processed for matte extraction, analyzes and processes the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics; then sends the position information to the client, so that the client determines the matte extraction region based on the position information and the selection operation on the candidate matte extraction regions in the original image, and sends the matte data of the matte extraction region to the server; finally, in response to receiving the matte data sent by the client, performs matte extraction processing on the original image. Compared with the prior art that determines candidate matte extraction regions by analyzing images based on pixel granularity, the generated candidate matte extraction regions are more in line with the user's matte extraction requirements, which helps to improve the matte extraction quality and reduce the occurrence of over-matte extraction and missed-matte extraction situations. On the other hand, when performing matte extraction processing on the original image based on the position information of the matte extraction region, the interaction times for determining the matte extraction region can be effectively reduced, and the matte extraction efficiency can be improved.

[0132] Based on the above embodiments, the embodiments of the present application also disclose an image processing system for implementing the above image processing method. As Figure 12 shown, the image processing system includes: a client and a server. The implementation manner of the image processing system is described below in conjunction with Figure 12 the interaction flowchart of the client and the server shown.

[0133] Step 1202, the client obtains the original image to be processed for matte extraction and displays the original image.

[0134] Step 1204, in response to the matte extraction operation on the original image, the client sends the original image to the preset server.

[0135] Step 1206, the server analyzes and processes the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics.

[0136] Step 1208, the server sends the position information to the client.

[0137] Step 1210, the client determines the matte extraction region based on the position information and the selection operation on the candidate matte extraction regions in the original image.

[0138] Step 1212, the client sends the matte data of the matte extraction region to the server.

[0139] Step 1214: The server, in response to receiving the masking data sent by the client, performs matte extraction on the original image.

[0140] For the specific implementation manners of the steps executed by the above client and server, refer to the relevant descriptions in the foregoing embodiments, and details are not described herein again.

[0141] In summary, for the image processing system disclosed in the embodiments of the present application, after the client obtains the original image to be subjected to matte extraction and sends the original image to the server, the server analyzes and processes the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics; then the server sends the position information to the client, and the client, in response to the user's selection operation on the candidate matte extraction regions in the original image and the position information, determines the matte extraction regions and sends the masking data of the matte extraction regions to the server; finally, the server performs matte extraction on the original image based on the masking data. Compared with the candidate matte extraction regions generated by analyzing the image based on the pixel granularity in the prior art, the candidate matte extraction regions of this system are more in line with the user's matte extraction requirements, which helps to improve the matte extraction quality and reduce the occurrence of over-matte extraction and missed-matte extraction. On the other hand, when performing matte extraction on the original image based on the position information of the matte extraction regions, the interaction times for determining the matte extraction regions can be effectively reduced, and the matte extraction efficiency can be improved.

[0142] Based on the above embodiments, the embodiments of the present application also disclose an image processing method, and the method includes: Step S1 to Step S5.

[0143] Step S1: Obtain the original image to be subjected to matte extraction and display the original image.

[0144] Step S2: In response to the matte extraction operation on the original image, analyze and process the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics.

[0145] Step S3: Based on the position information and the selection operation on the candidate matte extraction regions in the original image, determine the matte extraction regions.

[0146] Step S4: Based on the position information of the matte extraction regions, obtain the masking data of the matte extraction regions.

[0147] Step S5: Based on the masking data, perform matte extraction on the original image.

[0148] For the specific implementation manners of the above steps, reference may be made to the relevant descriptions in the foregoing embodiments, and details are not described herein again.

[0149] In summary, for the image processing method disclosed in the embodiments of the present application, after obtaining the original image to be subjected to matte extraction processing, the original image is displayed, and in response to a matte extraction operation on the original image, the original image is analyzed and processed based on a target granularity to obtain a candidate matte extraction area in the original image and the position information of the candidate matte extraction area, where the target granularity corresponds to the image content with scene semantics; then, based on the position information and a selection operation on the candidate matte extraction area in the original image, a matte extraction area is determined; based on the position information of the matte extraction area, matte data of the matte extraction area is obtained; and finally, matte extraction processing is performed on the original image based on the matte data. Compared with the candidate matte extraction areas generated by analyzing the image based on pixel granularity in the prior art, this method better meets the user's matte extraction requirements, helps to improve the matte extraction quality, and reduces the occurrence of over-matte extraction and missed-matte extraction. On the other hand, when performing matte extraction processing on the original image based on the position information of the matte extraction area, the interaction times for determining the matte extraction area can be effectively reduced, and the matte extraction efficiency can be improved.

[0150] Based on the above embodiments, the embodiments of the present application further disclose an online dressing system for implementing the above image processing method. The online dressing system includes: a client and a server, where

[0151] the client is configured to obtain an original image and display the original image;

[0152] the client is further configured to obtain dressing configuration information based on a user's selection operation, where the dressing configuration information includes one or more of the following: model information, retained clothing types, and generated image configuration information;

[0153] the client is further configured to send the original image to a preset server in response to an editing operation on the original image;

[0154] the server is configured to analyze and process the original image based on a target granularity to obtain a candidate matte extraction area in the original image and the position information of the candidate matte extraction area, and send the position information to the client, where the target granularity corresponds to the image content with scene semantics;

[0155] the client is further configured to determine a matte extraction area based on the position information and a selection operation on the candidate matte extraction area in the original image;

[0156] the client is further configured to send the matte data of the matte extraction area and the dressing configuration information to the server;

[0157] The server is further configured to generate a model dressing picture based on the original picture, the mask data, and the dressing configuration information, and send the model dressing picture to the client;

[0158] The client is further configured to display the model dressing picture.

[0159] Among them, the model information includes, but is not limited to, one or more of the following information: model gender, model style, etc.; the reserved clothing types include, but are not limited to: upper clothing, lower clothing, full-body clothing; the generated picture configuration information includes, but is not limited to: background style. Optionally, a configuration interface can be set on the client side for configuring the dressing configuration information based on user operations.

[0160] In some alternative embodiments, the user can upload an original picture containing clothing through the clothing picture upload entry set on the client interface, or browse and select an original picture containing clothing uploaded by the user in advance, such as a clothing dummy picture, through the clothing picture selection entry set on the client interface. Optionally, a picture editing entry can also be set on the client interface for triggering an editing operation on the original picture.

[0161] After detecting that the user triggers an editing operation on the original picture, the original picture is first sent to the server for analysis by the server. The specific implementation manner of the server analyzing and processing the original picture based on the target granularity to obtain the candidate matte regions in the original picture and the position information of the candidate matte regions can be referred to the relevant descriptions in the previous embodiments and will not be elaborated here. Among them, the target granularity corresponds to the picture content with scene semantics.

[0162] After that, the server sends the position information to the client, and the client determines the matte region based on the position information and the selection operation on the candidate matte regions in the original picture.

[0163] The specific implementation manner of the client determining the matte region based on the position information and the selection operation on the candidate matte regions in the original picture can be referred to the relevant descriptions in the previous embodiments and will not be elaborated here.

[0164] Next, the client sends the mask data of the mask added to the original picture corresponding to all the matte regions selected by the user and the dressing configuration information to the server.

[0165] The server performs matte extraction and image synthesis processing based on the mask data, the dressing configuration information, and the original picture sent by the client to obtain a model dressing picture.

[0166] Optionally, generating a model dressing picture based on the original picture, the matte data, and the dressing configuration information includes: performing matte extraction on the original picture based on the matte data to obtain an extracted image; and generating the dressing configuration information based on the extracted image and the dressing configuration information. Taking the hat, top, and skirt shown in Figure 9 as an example, the matte data is the matte added by the client in the hat, top, and skirt areas after the user selects the hat, top, and skirt areas in the original picture as the matte extraction areas. The server first performs matte extraction on the original picture based on this matte data to obtain the images of the hat, top, and skirt; then, the server generates a model dressing picture in which the selected model wears the top and skirt in the original picture and wears the hat in the original picture according to the selected model gender and model style.

[0167] After that, the server sends the generated model dressing picture to the client, and the client displays the model dressing picture.

[0168] For the specific implementation manners of the client and the server to execute the above steps, refer to the relevant descriptions in the foregoing embodiments, which will not be elaborated here.

[0169] In summary, the online dressing system disclosed in the embodiments of the present application obtains an original picture to be subjected to matte extraction and configures dressing configuration information on the client, and then sends the original picture to the server. The server analyzes and processes the original picture based on the target granularity to obtain candidate matte extraction areas in the original picture and the position information of the candidate matte extraction areas, where the target granularity corresponds to picture content with scene semantics; then the server sends the position information to the client, and the client determines the matte extraction area in response to the user's selection operation on the candidate matte extraction areas in the original picture and the position information, and sends the matte data of the matte extraction area to the server; finally, the server performs matte extraction on the original picture based on the matte data and the original picture, and then synthesizes the obtained clothing images and dressing configuration information to obtain a picture of a specified style model wearing some or all of the clothing in the original picture, which can improve the quality of the obtained clothing images, reduce the occurrence of over-matte extraction and missed matte extraction, and thus improve the quality of the generated model dressing picture based on the clothing images. Moreover, this system effectively reduces the number of interactions for determining the matte extraction area, thereby reducing the number of interactions for generating the model dressing picture and improving the matte extraction efficiency.

[0170] It should be noted that, for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this application are not limited by the described action sequence, because according to the embodiments of this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of this application.

[0171] Based on the above embodiments, this embodiment further provides an image processing device, which is applied to a client. The device includes:

[0172] An original image acquisition and display module, configured to acquire an original image to be processed by matting and display the original image;

[0173] An image analysis and processing module, configured to, in response to a matting operation on the original image, send the original image to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matting regions in the original image and position information of the candidate matting regions, where the target granularity corresponds to image content with scene semantics;

[0174] A matting region determination module, configured to determine a matting region based on the position information and a selection operation on the candidate matting regions in the original image;

[0175] A matting processing module, configured to perform matting processing on the original image based on the position information of the matting region.

[0176] Optionally, the determining a matting region based on the position information and a selection operation on the candidate matting regions in the original image includes:

[0177] In response to detecting a region selection operation on the candidate matting regions in the original image, determining a first target candidate matting region targeted by the region selection operation according to the image position corresponding to the region selection operation and the position information of the candidate matting regions;

[0178] Updating the current selection state of the first target candidate matting region;

[0179] Performing color filling processing on the first target candidate matting region based on the updated selection state;

[0180] Updating the matting region based on the updated selection state of the first target candidate matting region.

[0181] Optionally, the selection state includes: a selected state indicating that the first target candidate matte region has been selected as a matte region, and an unselected state indicating that the first target candidate matte region has not been selected as a matte region. Updating the current selection state of the first target candidate matte region includes:

[0182] In response to the current selection state of the first target candidate matte region being the selected state, updating the selection state to the unselected state;

[0183] In response to the current selection state of the first target candidate matte region being the unselected state, updating the selection state to the selected state.

[0184] Optionally, the selection state includes: a selected state indicating that the first target candidate matte region has been selected as a matte region, and an unselected state indicating that the first target candidate matte region has not been selected as a matte region. Updating the matte region based on the updated selection state of the first target candidate matte region includes:

[0185] In response to the updated selection state of the first target candidate matte region being the selected state, adding the first target candidate matte region as a matte region;

[0186] In response to the updated selection state of the first target candidate matte region being the unselected state, removing the first target candidate matte region from the matte region.

[0187] Optionally, the selection state includes: a selected state indicating that the first target candidate matte region has been selected as a matte region, and an unselected state indicating that the first target candidate matte region has not been selected as a matte region. Performing color filling processing on the first target candidate matte region based on the updated selection state includes:

[0188] In response to the updated selection state of the first target candidate matte region being the selected state, filling the first target candidate matte region with a first preset color;

[0189] In response to the updated selection state of the first target candidate matte region being the unselected state, clearing the first preset color filled in the first target candidate matte region.

[0190] Optionally, after sending the original image to a preset server so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions, it further includes:

[0191] On the interface that displays the original picture, a preview drawing board for the matte extraction area is displayed in contrast.

[0192] After determining the matte extraction area based on the position information and the selection operation on the candidate matte extraction area in the original picture, it further includes:

[0193] Based on the position information, the matte extraction area is drawn in real time on the preview drawing board.

[0194] Optionally, after responding to the matte extraction operation on the original picture, sending the original picture to a preset server, so that the preset server analyzes and processes the original picture based on the target granularity to obtain the candidate matte extraction areas in the original picture and the position information of the candidate matte extraction areas, it further includes:

[0195] In response to detecting a regional preview operation on the candidate matte extraction area in the original picture, according to the picture position corresponding to the regional preview operation and the position information of the candidate matte extraction area, determining the second target candidate matte extraction area targeted by the regional preview operation;

[0196] Based on the current selection status of the second target candidate matte extraction area, the second target candidate matte extraction area is highlighted in real time with a corresponding preset color.

[0197] Optionally, the selection status includes: a selected state indicating that the second target candidate matte extraction area has been selected as the matte extraction area, and an unselected state indicating that the second target candidate matte extraction area has not been selected as the matte extraction area. Based on the current selection status of the second target candidate matte extraction area, highlighting the second target candidate matte extraction area in real time with a corresponding preset color includes:

[0198] In response to the current selection status of the second target candidate matte extraction area being the selected state, highlighting the second target candidate matte extraction area in real time with a second preset color;

[0199] In response to the current selection status of the second target candidate matte extraction area being the unselected state, highlighting the second target candidate matte extraction area in real time with a third preset color.

[0200] Optionally, sending the original picture to the preset server, so that the preset server analyzes and processes the original picture based on the target granularity to obtain the candidate matte extraction areas in the original picture and the position information of the candidate matte extraction areas, includes:

[0201] Send the original image to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and a pixel point matrix of the candidate matte regions;

[0202] Perform region contour detection based on the pixel point matrix to obtain a vector of the edge contour in the original image;

[0203] Perform region splitting processing on the vector to obtain a vector for drawing the edge contour of each candidate matte region, which serves as the position information of the corresponding candidate matte region

[0204] The image processing device disclosed in the embodiments of the present application is used to implement the above-mentioned image processing method. For the specific implementation manners of the modules of the device, refer to the specific implementation manners of the corresponding steps in the foregoing method embodiments, which will not be elaborated here.

[0205] In summary, the image processing device disclosed in the embodiments of the present application obtains an original image to be matte processed and displays the original image. Then, in response to a matte operation on the original image, the original image is sent to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and the position information of the candidate matte regions, where the target granularity corresponds to image content with scene semantics, so that the matte region determined based on the position information and the selection operation of the candidate matte regions in the original image is more in line with the user's matte requirements, which helps to improve the matte quality and reduce the occurrence of over-matting and under-matting situations. On the other hand, when performing matte processing on the original image based on the position information of the matte region, the number of interactions for determining the matte region can be effectively reduced, and the matte efficiency can be improved.

[0206] Based on the above embodiments, this embodiment further provides an image processing device applied to a server. The device includes:

[0207] An image analysis and processing module, configured to, in response to receiving an original image to be matte processed sent by a client, analyze and process the original image based on a target granularity to obtain candidate matte regions in the original image and the position information of the candidate matte regions, where the target granularity corresponds to image content with scene semantics;

[0208] A position information sending module, configured to send the position information to the client, so that the client determines a matte region based on the position information and the selection operation of the candidate matte regions in the original image, and sends matte data of the matte region to the server;

[0209] The matte extraction processing module is configured to perform matte extraction processing on the original image in response to receiving the matte data sent by the client.

[0210] The image processing device disclosed in the embodiments of the present application is used to implement the above-mentioned image processing method. For the specific implementation manners of the modules of the device, refer to the specific implementation manners of the corresponding steps in the foregoing method embodiments, which will not be elaborated herein.

[0211] In summary, the image processing device disclosed in the embodiments of the present application, after receiving the original image to be matte-extracted sent by the client, analyzes and processes the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics; then sends the position information to the client, so that the client determines the matte extraction region based on the position information and the selection operation on the candidate matte extraction regions in the original image, and sends the matte data of the matte extraction region to the server; finally, in response to receiving the matte data sent by the client, performs matte extraction processing on the original image. Compared with the prior art that determines candidate matte extraction regions by analyzing images based on pixel granularity, the generated candidate matte extraction regions are more in line with the user's matte extraction requirements, which helps to improve the matte extraction quality and reduce the occurrence of over-matte and under-matte situations. On the other hand, when performing matte extraction processing on the original image based on the position information of the matte extraction region, the interaction times for determining the matte extraction region can be effectively reduced, and the matte extraction efficiency can be improved.

[0212] Based on the above embodiments, the present embodiment further provides an image processing device, which includes:

[0213] The original image acquisition and display module is configured to acquire the original image to be matte-extracted and display the original image;

[0214] The image analysis module is configured to, in response to a matte extraction operation on the original image, analyze and process the original image based on the target granularity to obtain the candidate matte extraction regions in the original image and the position information of the candidate matte extraction regions, where the target granularity corresponds to the image content with scene semantics;

[0215] The matte extraction region determination module is configured to determine the matte extraction region based on the position information and the selection operation on the candidate matte extraction regions in the original image;

[0216] The matte data acquisition module is configured to obtain the matte data of the matte extraction region based on the position information of the matte extraction region;

[0217] The matte extraction processing module is configured to perform matte extraction processing on the original image based on the matte data.

[0218] The picture processing device disclosed in the embodiments of the present application is used to implement the above-mentioned picture processing method. For the specific implementation manners of the modules of the device, refer to the specific implementation manners of the corresponding steps in the foregoing method embodiments, which will not be elaborated here.

[0219] The embodiments of the present application also provide a non-volatile readable storage medium, in which one or more modules (programs) are stored. When the one or more modules are applied to a device, the device can be caused to execute the instructions (instructions) of the various method steps in the embodiments of the present application.

[0220] The embodiments of the present application also provide a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method as described in the embodiments of the present application.

[0221] The embodiments of the present application also provide an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method as described in the embodiments of the present application. In the embodiments of the present application, the electronic device includes devices such as servers and terminal devices.

[0222] The embodiments of the present application also disclose a computer program product, including a computer program / computer-executable instructions, characterized in that when the computer program / computer-executable instructions are executed by a processor in an electronic device, the method as described in the embodiments of the present application is implemented.

[0223] The embodiments of the present disclosure can be implemented as a device configured as desired using any suitable hardware, firmware, software, or any combination thereof. The device may include electronic devices such as servers (clusters) and terminals. Figure 13 Exemplary device 1300 that can be used to implement the various embodiments described in the present application is schematically shown.

[0224] For one embodiment, Figure 13 Exemplary device 1300 is shown, which has one or more processors 1302, a control module (chipset) 1304 coupled to at least one of the (one or more) processors 1302, a memory 1306 coupled to the control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1304.

[0225] The processor 1302 may include one or more single-core or multi-core processors, and the processor 1302 may include any combination of general-purpose processors or dedicated processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1300 can act as devices such as the server, terminal, etc. described in the embodiments of the present application.

[0226] In some embodiments, the device 1300 may include one or more computer-readable media (such as the memory 1306 or the NVM / storage device 1308) having instructions 1314, and one or more processors 1302 combined with the one or more computer-readable media and configured to execute the instructions 1314 to implement modules so as to perform the actions described in the present disclosure.

[0227] For one embodiment, the control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the (one or more) processors 1302 and / or any suitable device or component communicating with the control module 1304.

[0228] The control module 1304 may include a memory controller module to provide an interface to the memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0229] The memory 1306 may be used, for example, to load and store data and / or instructions 1314 for the device 1300. For one embodiment, the memory 1306 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1306 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0230] For one embodiment, the control module 1304 may include one or more input / output controllers to provide interfaces to the NVM / storage device 1308 and the (one or more) input / output devices 1310.

[0231] For example, the NVM / storage device 1308 may be used to store data and / or instructions 1314. The NVM / storage device 1308 may include any suitable non-volatile memory (such as flash memory) and / or may include any suitable (one or more) non-volatile storage devices (such as one or more hard disk drives (HDDs), one or more optical discs (CD) drives, and / or one or more digital versatile discs (DVD) drives).

[0232] The NVM / storage device 1308 may include storage resources that are part of the device on which the apparatus 1300 is installed, or it may be accessible to the device without being part of the device. For example, the NVM / storage device 1308 may be accessed via the network through the input / output device(s) 1310.

[0233] (The one or more) input / output devices 1310 may provide an interface for the apparatus 1300 to communicate with any other suitable device. The input / output device 1310 may include a communication component, an audio component, a sensor component, etc. The network interface 1312 may provide an interface for the apparatus 1300 to communicate through one or more networks. The apparatus 1300 may wirelessly communicate with one or more components of a wireless network according to any one of one or more wireless network standards and / or protocols, such as accessing a wireless network based on a communication standard, such as Bluetooth, WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0234] For one embodiment, at least one of the processor(s) 1302 may be logically encapsulated with one or more controllers of the control module 1304 (e.g., a memory controller module). For one embodiment, at least one of the processor(s) 1302 may be logically encapsulated with one or more controllers of the control module 1304 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1302 may be logically integrated with one or more controllers of the control module 1304 on the same die. For one embodiment, at least one of the processor(s) 1302 may be logically integrated with one or more controllers of the control module 1304 on the same die to form a system-on-chip (SoC).

[0235] In various embodiments, the apparatus 1300 may be, but is not limited to, a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.), etc. terminal devices. In various embodiments, the apparatus 1300 may have more or fewer components and / or a different architecture. For example, in some embodiments, the apparatus 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application specific integrated circuit (ASIC), and a speaker.

[0236] Among them, a main control chip may be used as the processor or the control module in the detection device. Sensor data, location information, etc. are stored in the memory or the NVM / storage device. The sensor group may be used as an input / output device, and the communication interface may include a network interface.

[0237] An embodiment of the present application further provides an electronic device, including: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to execute one or more of the methods in the embodiments of the present application. In the embodiments of the present application, various data can be stored in the memory, such as target files, file-application association data, and other various data, and can also include user behavior data, etc., thereby providing a data basis for various processes.

[0238] An embodiment of the present application further provides one or more machine-readable media storing executable code thereon, which, when executed, causes a processor to execute one or more of the methods in the embodiments of the present application.

[0239] For the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0240] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and for the same or similar parts among the embodiments, reference can be made to each other.

[0241] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0242] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing terminal devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0243] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide for implementing the process Figure 1 One process or multiple processes and / or blocks Figure 1 Steps for the functions specified in one block or multiple blocks.

[0244] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0245] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0246] The above has provided a detailed introduction to a picture processing method, a picture processing system, an online dressing system, an electronic device, a storage medium and a computer program product provided by the present application. Specific examples are used in this text to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An image processing method, applied to a client, characterized in that, The method includes: Obtain an original image to be processed for matte extraction, and display the original image; In response to a matte extraction operation on the original image, send the original image to a preset server, so that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions. Wherein, the target granularity corresponds to the image content with scene semantics, the target granularity is determined according to the smallest component with semantic meaning of the target subject, and the target granularity is obtained by the preset segmentation model through parameter tuning methods such as controlling the density of sampling points, setting the intersection-over-union threshold, and returning the minimum area of the mask for learning the training data. The candidate matte regions obtained by the preset segmentation model are selected from various parts of clothing and various parts of the human body. The sending the original image to the preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions includes: sending the original image to the preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain candidate matte regions in the original image and a pixel point matrix of the candidate matte regions; performing region contour detection based on the pixel point matrix to obtain a vector of the edge contour in the original image; performing region splitting processing on the vector to obtain a vector for drawing the edge contour of each candidate matte region as the position information of the corresponding candidate matte region; Based on the position information and the selection operation on the candidate matte regions in the original image, determine the matte region; Perform matte extraction processing on the original image based on the position information of the matte region.

2. The method according to claim 1, characterized in that, The determining the matte region based on the position information and the selection operation on the candidate matte regions in the original image includes: In response to detecting a region selection operation on the candidate matte regions in the original image, determine a first target candidate matte region targeted by the region selection operation according to the image position corresponding to the region selection operation and the position information of the candidate matte regions; Update the current selection state of the first target candidate matte region; Perform color filling processing on the first target candidate matte region based on the updated selection state; Update the matte region based on the updated selection state of the first target candidate matte region.

3. The method according to claim 2, wherein The selection state includes: a selected state indicating that the first target candidate matte region has been selected as the matte region, and an unselected state indicating that the first target candidate matte region has not been selected as the matte region. The updating the current selection state of the first target candidate matte region includes: In response to the current selection state of the first target candidate matte region being the selected state, update the selection state to the unselected state; In response to the current selection state of the first target candidate matte region being the unselected state, update the selection state to the selected state.

4. The method according to claim 2, wherein The selection status includes: a selected status indicating that the first target candidate matte region has been selected as the matte region, and an unselected status indicating that the first target candidate matte region has not been selected as the matte region. Updating the matte region based on the updated selection status of the first target candidate matte region includes: In response to the updated selection status of the first target candidate matte region being the selected status, adding the first target candidate matte region as the matte region; In response to the updated selection status of the first target candidate matte region being the unselected status, removing the first target candidate matte region from the matte region.

5. The method according to claim 2, wherein The selection status includes: a selected status indicating that the first target candidate matte region has been selected as the matte region, and an unselected status indicating that the first target candidate matte region has not been selected as the matte region. Performing color filling processing on the first target candidate matte region based on the updated selection status includes: In response to the updated selection status of the first target candidate matte region being the selected status, filling the first target candidate matte region with a first preset color; In response to the updated selection status of the first target candidate matte region being the unselected status, clearing the first preset color filled in the first target candidate matte region.

6. The method according to claim 1, characterized in that After sending the original image to a preset server, such that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions, it further includes: On the interface for displaying the original image, a preview palette for displaying the matte region is presented in contrast; After determining the matte region based on the position information and a selection operation on the candidate matte regions in the original image, it further includes: Based on the position information, the matte region is drawn in real time on the preview palette.

7. The method according to claim 1, characterized in that, After responding to a matte operation on the original image by sending the original image to a preset server, such that the preset server analyzes and processes the original image based on a target granularity to obtain candidate matte regions in the original image and position information of the candidate matte regions, it further includes: In response to detecting a region preview operation on the candidate matte regions in the original image, determining a second target candidate matte region targeted by the region preview operation according to the image position corresponding to the region preview operation and the position information of the candidate matte regions; Based on the current selection status of the second target candidate matte region, the second target candidate matte region is highlighted in real time with a corresponding preset color.

8. The method according to claim 7, wherein The selection status includes: a selected status indicating that the second target candidate matte region has been selected as the matte region, and an unselected status indicating that the second target candidate matte region has not been selected as the matte region. Highlighting the second target candidate matte region in real time with a corresponding preset color based on the current selection status of the second target candidate matte region includes: In response to the current selection status of the second target candidate matte region being the selected status, the second target candidate matte region is highlighted in real time with a second preset color; In response to the current selection status of the second target candidate matte region being the unselected status, the second target candidate matte region is highlighted in real time with a third preset color.

9. A method for image processing, applied to a server, characterized in that, The method includes: In response to receiving the original image to be matte processed sent by the client, the original image is analyzed and processed based on the target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions. Among them, the target granularity corresponds to the image content with scene semantics, the target granularity is determined according to the smallest component with semantic meaning of the target subject, the target granularity is obtained by the preset segmentation model through parameter tuning methods such as controlling the density of sampling points, setting the intersection over union threshold, and returning the minimum area of the mask for learning the training data. The candidate matte regions obtained by the preset segmentation model are selected from various parts of clothing and various parts of the human body. Sending the original image to the preset server, so that the preset server analyzes and processes the original image based on the target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions, including: sending the original image to the preset server, so that the preset server analyzes and processes the original image based on the target granularity to obtain the candidate matte regions in the original image and the pixel point matrix of the candidate matte regions; performing region contour detection based on the pixel point matrix to obtain the vector of the edge contour in the original image; performing region splitting processing on the vector to obtain the vector for drawing the edge contour of each candidate matte region, as the position information of the corresponding candidate matte region; Sending the position information to the client, so that the client determines the matte region based on the position information and the selection operation of the candidate matte regions in the original image, and sends the matte data of the matte region to the server; In response to receiving the matte data sent by the client, performing matte processing on the original image.

10. A method for image processing, characterized in that, The method includes: Obtaining the original image to be matte processed and displaying the original image; In response to a matte extraction operation on the original image, the original image is analyzed and processed based on a target granularity to obtain candidate matte regions in the original image and the position information of the candidate matte regions. Herein, the target granularity corresponds to the image content with scene semantics, the target granularity is determined according to the smallest component with semantic meaning of the target subject, and the target granularity is obtained after the preset segmentation model learns the training data by means of parameter tuning such as controlling the density of sampling points, setting the intersection over union threshold, and returning the minimum area of the mask. The candidate matte regions obtained by the preset segmentation model are selected from various parts of clothing and various parts of the human body. Sending the original image to a preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain candidate matte regions in the original image and the position information of the candidate matte regions includes: sending the original image to the preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain the candidate matte regions in the original image and the pixel point matrix of the candidate matte regions; performing region contour detection based on the pixel point matrix to obtain the vector of the edge contour in the original image; performing region splitting processing on the vector to obtain the vector for drawing the edge contour of each candidate matte region as the position information of the corresponding candidate matte region; Based on the position information and the selection operation on the candidate matte regions in the original image, the matte region is determined; Based on the position information of the matte region, the matte data of the matte region is obtained; Based on the matte data, matte extraction processing is performed on the original image.

11. An image processing system, characterized in that, The system includes: a client and a server, wherein, The client is used to execute the image processing method according to any one of claims 1 to 8; The server is used to execute the image processing method according to claim 9.

12. An online dressing system, characterized in that, The system includes: a client and a server, wherein, The client is used to obtain the original image and display the original image; The client is further used to obtain dressing configuration information based on the user's selection operation, wherein the dressing configuration information includes one or more of the following: model information, retained clothing type, generated image configuration information; The client is further used to send the original image to a preset server in response to an editing operation on the original image; The server is used to analyze and process the original image based on the target granularity, obtain the candidate matte regions in the original image and the position information of the candidate matte regions, and send the position information to the client. Among them, the target granularity corresponds to the image content with scene semantics, the target granularity is determined according to the smallest component with semantic meaning of the target subject, and the target granularity is obtained by the preset segmentation model through parameter tuning methods such as controlling the density of sampling points, setting the intersection over union threshold, and returning the minimum area of the mask. The candidate matte regions obtained by the preset segmentation model are selected from various parts of the clothing and various parts of the human body. Sending the original image to the preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain the candidate matte regions in the original image and the position information of the candidate matte regions includes: sending the original image to the preset server so that the preset server analyzes and processes the original image based on the target granularity to obtain the candidate matte regions in the original image and the pixel point matrix of the candidate matte regions; performing region contour detection based on the pixel point matrix to obtain the vector of the edge contour in the original image; performing region splitting processing on the vector to obtain the vector for drawing the edge contour of each candidate matte region as the position information of the corresponding candidate matte region; The client is further used to determine the matte region based on the position information and the selection operation of the candidate matte region in the original image; The client is further used to send the matte data of the matte region and the dressing configuration information to the server; The server is further used to generate a model dressing image based on the original image, the matte data and the dressing configuration information, and send the model dressing image to the client; The client is further used to display the model dressing image.

13. An electronic device, characterized in that, It includes: a processor and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method according to any one of claims 1-10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the method according to any one of claims 1-10.

15. A computer program product, comprising a computer program / computer-executable instructions, characterized in that, When the computer program / computer executable instructions are executed by the processor in the electronic device, the method according to any one of claims 1-10 is implemented.

Images