Product publishing method and image processing method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a product publishing method and an image processing method, device, equipment and storage medium. The method includes: obtaining an input image, which contains a foreground object; generating a first mask image corresponding to the image; setting the pixels corresponding to the foreground object in the first mask image to a first grayscale value range, and setting the pixels corresponding to the non-foreground object in the first mask image to a second grayscale value range; and determining the contour corresponding to the foreground object in the image according to the grayscale value difference between the foreground object and the non-foreground object. So that the user can interact with the foreground object based on the contour recognition result of the foreground object. When this solution is adopted in a certain product, the interactive mode of the image provided by it is expanded, and the user's stickiness to the product is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a commodity publishing method and an image processing method, device, equipment and storage medium. Background Art

[0002] With the development of Internet technology, people can obtain all kinds of items and information they need through the Internet without leaving home. For example, they can buy various goods online through various shopping applications (APPs) and watch live broadcasts online through live broadcast APPs.

[0003] When users use these apps, a lot of content can be displayed on the app interface. For example, a certain interface of a shopping app can display pictures and videos of products; the interface of a live broadcast app can display video footage of the host.

[0004] Currently, for the pictures and video images displayed on the interface, users can either only watch but not interact, or the forms of interaction are limited. Summary of the invention

[0005] The embodiments of the present invention provide a commodity publishing method and an image processing method, device, equipment and storage medium, which can expand the interactive mode of images and improve user stickiness.

[0006] In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising:

[0007] Acquire an image, wherein the image contains a foreground object;

[0008] generating a first mask image corresponding to the image;

[0009] Setting pixels corresponding to the foreground object in the first mask image to a first grayscale value range, and setting pixels corresponding to non-foreground objects in the first mask image to a second grayscale value range;

[0010] According to the difference in grayscale values ​​between the foreground object and the non-foreground object, a contour corresponding to the foreground object in the image is determined.

[0011] In a second aspect, an embodiment of the present invention provides an image processing device, the device comprising:

[0012] An acquisition module, used for acquiring an image, wherein the image contains a foreground object;

[0013] A generating module, used for generating a first mask image corresponding to the image;

[0014] A setting module, used to set the pixels corresponding to the foreground object in the first mask image to a first gray value range, and set the pixels corresponding to the non-foreground object in the first mask image to a second gray value range;

[0015] The determination module is used to determine the contour corresponding to the foreground object in the image according to the gray value difference between the foreground object and the non-foreground object.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the image processing method described in the first aspect.

[0017] An embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image processing method described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present invention provides an image processing method, the method comprising:

[0019] Acquire an image, wherein the image contains a foreground object;

[0020] generating a mask image corresponding to the image;

[0021] Setting the pixels corresponding to the foreground object in the mask image to a first grayscale value range, and setting the pixels corresponding to the non-foreground object in the mask image to a second grayscale value range;

[0022] The foreground object is cut out from the image according to a grayscale value difference between the foreground object and the non-foreground object.

[0023] In a fifth aspect, an embodiment of the present invention provides an image processing device, the device comprising:

[0024] An acquisition module, used for acquiring an image, wherein the image contains a foreground object;

[0025] A generating module, used for generating a mask image corresponding to the image;

[0026] A setting module, used to set the pixels corresponding to the foreground object in the mask image to a first gray value range, and set the pixels corresponding to the non-foreground object in the mask image to a second gray value range;

[0027] The screenshot module is used to capture the foreground object in the image according to the gray value difference between the foreground object and the non-foreground object.

[0028] In a sixth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the image processing method as described in the fourth aspect.

[0029] An embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image processing method described in the fourth aspect.

[0030] In a seventh aspect, an embodiment of the present invention provides an image processing method, the method comprising:

[0031] Acquire an image, wherein the image contains a foreground object;

[0032] generating a mask image corresponding to the image;

[0033] Setting the pixels corresponding to the foreground object in the mask image to a first grayscale value range, and setting the pixels corresponding to the non-foreground object in the mask image to a second grayscale value range;

[0034] According to the gray value difference between the foreground object and the non-foreground object, a first layer corresponding to the foreground object and a second layer corresponding to the non-foreground object are generated.

[0035] In an eighth aspect, an embodiment of the present invention provides an image processing device, the device comprising:

[0036] An acquisition module, used for acquiring an image, wherein the image contains a foreground object;

[0037] A generating module, used for generating a mask image corresponding to the image;

[0038] A setting module, used to set the pixels corresponding to the foreground object in the mask image to a first gray value range, and set the pixels corresponding to the non-foreground object in the mask image to a second gray value range;

[0039] The layering module is used to generate a first layer corresponding to the foreground object and a second layer corresponding to the non-foreground object according to the gray value difference between the foreground object and the non-foreground object.

[0040] In a ninth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the image processing method as described in the seventh aspect.

[0041] An embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image processing method described in the seventh aspect.

[0042] In a tenth aspect, an embodiment of the present invention provides a product publishing method, the method comprising:

[0043] Acquire a first image, wherein the first image contains a product to be released;

[0044] generating a mask image corresponding to the first image;

[0045] Set the pixels corresponding to the product in the mask image to a first grayscale value range, and set the pixels corresponding to the background objects in the first image in the mask image to a second grayscale value range;

[0046] Determining a contour corresponding to the product in the first image according to a grayscale value difference between the product and the background object;

[0047] A second image corresponding to the product is generated according to the outline of the product in the first image and a product publishing template, so as to publish the product through the second image.

[0048] In an eleventh aspect, an embodiment of the present invention provides a commodity publishing device, the device comprising:

[0049] An acquisition module, used to acquire a first image, wherein the first image contains a product to be released;

[0050] A generating module, used for generating a mask image corresponding to the first image;

[0051] A setting module, used to set the pixels corresponding to the commodity in the mask image to a first gray value range, and set the pixels corresponding to the background objects in the first image in the mask image to a second gray value range;

[0052] A determination module, configured to determine a contour corresponding to the product in the first image according to a difference in grayscale values ​​between the product and the background object;

[0053] The publishing module is used to generate a second image corresponding to the product according to the outline of the product in the first image and a product publishing template, so as to publish the product through the second image.

[0054] In the twelfth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the commodity publishing method as described in the tenth aspect.

[0055] An embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the commodity publishing method as described in the tenth aspect.

[0056] In the embodiment of the present invention, for any image, the image may contain a foreground object and a background object. To support the interaction between the user and the foreground object, first, a first mask image corresponding to the image is generated, wherein if the image contains a foreground object, the pixels corresponding to the foreground object in the first mask image are set to a first grayscale value range, and the pixels corresponding to the non-foreground object in the first mask image are set to a second grayscale value range. That is, by generating the first mask image, it is possible to identify whether the image contains a foreground object that allows the user to interact, and in the case of determining that the foreground object is contained, the foreground object and the background object in the image are binarized in the first mask image to obtain the pixel positions corresponding to the foreground object in the image. Thus, the contour corresponding to the foreground object in the image can be determined according to the first mask image, that is, the image area defined by the contour corresponds to the foreground object. Finally, based on the contour recognition result corresponding to the foreground object in the image, multiple interactive processing can be performed on the image so that the user can interact with the foreground object. When the above solution is adopted in a certain product, the interactive mode of the image provided by it is expanded, and the user's stickiness to the product is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 A flowchart of an image processing method provided by an embodiment of the present invention;

[0059] Figure 2A schematic diagram of the principle of determining the contour of a foreground object in an image based on a mask image corresponding to the image;

[0060] Figure 3 is a schematic diagram of an image without a foreground object and a mask image corresponding to the image;

[0061] Figure 4 A schematic diagram of the main working principle of a semantic segmentation model provided by an embodiment of the present invention;

[0062] Figure 5 A schematic diagram of interface changes of an image interaction method provided by an embodiment of the present invention;

[0063] Figure 6 A schematic diagram of an application scenario of an image processing method provided by an embodiment of the present invention;

[0064] Figure 7 A schematic diagram of interface changes of another image interaction method provided by an embodiment of the present invention;

[0065] Figure 8 A schematic diagram of interface changes of another image interaction method provided by an embodiment of the present invention;

[0066] Fig. 9 A schematic diagram of a training process of a semantic segmentation model provided in an embodiment of the present invention;

[0067] Fig.10 A schematic diagram of the effect of an image enhancement processing method provided by an embodiment of the present invention;

[0068] Fig.11 A schematic diagram of another image enhancement processing method provided by an embodiment of the present invention;

[0069] Fig.12 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;

[0070] Fig.13 For Fig.12 A schematic structural diagram of an electronic device corresponding to the image processing device provided in the illustrated embodiment. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0073] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0074] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0075] The image processing method provided in the embodiment of the present invention can be executed by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, etc., or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server.

[0076] The image processing method provided by the embodiment of the present invention can be applied to application scenarios such as online shopping. Taking the online shopping application scenario as an example, at this time, the image processing method can process images and videos uploaded by merchants for introducing products. It can be understood that a video is also composed of several frames of images, so the processing of a video can be considered as the processing of several frames of images constituting the video.

[0077] Figure 1 A flowchart of an image processing method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the method comprises the following steps:

[0078] 101. Acquire an image, wherein the image contains a foreground object.

[0079] 102. Generate a first mask image corresponding to the image, set pixels corresponding to foreground objects in the first mask image to a first grayscale value range, and set pixels corresponding to non-foreground objects in the first mask image to a second grayscale value range.

[0080] 103. Determine the contour corresponding to the foreground object in the image according to the difference in grayscale values ​​between the foreground object and the non-foreground object.

[0081] Taking online shopping as an example, the above image can be an image uploaded by a merchant to introduce a product. For example, a merchant's product is a sweater, and the merchant uploaded one or more pictures of a model wearing the sweater. In this case, each of the pictures can be used as the above image. Or, the merchant uploaded a video in which a model walked on the street wearing the sweater. In this case, each frame of the image sampled in the video can be used as the above image.

[0082] In actual applications, taking a user as a consumer as an example, assuming that the user searches and browses products online through his own terminal device (such as a mobile phone or computer), if the user clicks on the link address of the sweater and then goes to the details page of the sweater, at this time, the execution of the image processing method provided in this embodiment can be triggered, that is, the above-mentioned image contained in the details page is used as the input of the image processing method, and the above-mentioned steps are performed on the image.

[0083] Of course, the timing for executing the image processing method is not limited thereto. For example, the method may be triggered to execute when a merchant uploads a product image.

[0084] The core idea of ​​the image processing method is summarized as follows: The purpose of the image processing method is to identify whether the image contains a foreground object and the corresponding contour of the foreground object in the image, so that based on the contour recognition result of the foreground object, the user can interact with the foreground object in the image.

[0085] For the convenience of description, the above image is described as an input image, and whether the input image contains a foreground object and the corresponding contour of the foreground object in the input image can be identified in the following manner: a first mask image (Mask image) corresponding to the input image is generated. Specifically, if it is identified that the input image contains a foreground object, the pixels corresponding to the foreground object in the first mask image are set to the first grayscale value range, and the pixels corresponding to the non-foreground object (that is, it can be considered as a background object) in the first mask image are set to the second grayscale value range. Of course, if it is identified that the input image does not contain a foreground object, all pixels in the first mask image are set to the second grayscale value range.

[0086] The first grayscale value range and the second grayscale value range may be a specific grayscale value or a grayscale value interval. For example, the first grayscale value range is a grayscale value (0) corresponding to black, and the second grayscale value range is a grayscale value (255) corresponding to white. When the first grayscale value range and the second grayscale value range are a value interval, it can be considered that the input image is converted from a color image to a grayscale image, and the first grayscale value range and the second grayscale value range can clearly distinguish foreground objects from non-foreground objects.

[0087] Of course, for the convenience of processing, when the first gray value range and the second gray value range are a value interval, a suitable threshold can be set to convert the first mask image into a black and white binary image. The threshold is determined according to the first gray value range and the second gray value range, so that, for example, the gray value of the first gray value range is binarized to 0, and the gray value of the second gray value range is binarized to 255.

[0088] In addition, the size of the initially generated first mask image may be equal to or different from the size of the input image. When they are different, the size of the first mask image may be adjusted to be equal to the size of the input image by scaling.

[0089] It is worth noting that, when the size of the first mask image can be equal to the size of the input image, when it is recognized that the input image contains a foreground object, it can be considered that the pixels corresponding to the foreground object in the first mask image and the pixels corresponding to the foreground object in the input image are the same in position.

[0090] Based on this, after obtaining the binary first mask image, if the first mask image contains two grayscale values, it means that the input image contains a foreground object. At this time, the corresponding contour of the foreground object in the input image can be determined based on the difference between the two grayscale value ranges in the first mask image.

[0091] In addition, since the pixels corresponding to the foreground object in the first mask image are set to the first grayscale value range, and the pixels corresponding to the non-foreground object in the first mask image are set to the second grayscale value range, it can be considered that each pixel in the first mask image whose grayscale value corresponds to the first grayscale value range corresponds to the foreground object, and thus, optionally, a color rendering suggestion for the foreground object that matches the background color (i.e., the color corresponding to the non-foreground object in the input image) can also be provided in the first mask image, that is, each pixel corresponding to the foreground object can be rendered with the suggested color. In practical applications, a color matching relationship can be pre-set to give the above suggestion based on the matching relationship.

[0092] For ease of description, the two grayscale values ​​after binarization are referred to as grayscale value 1 (corresponding to black) and grayscale value 2 (corresponding to white) in the following text.

[0093] Among them, optionally, according to the difference in grayscale values ​​between the foreground object and the non-foreground object, the process of determining the contour corresponding to the foreground object in the input image can be implemented as follows: extracting the contour of the figure composed of these pixels with a grayscale value of 1 (i.e. corresponding to the foreground object) in the first mask image through a set contour extraction algorithm (such as the Open CV algorithm), at this time, the multiple pixel positions constituting the contour are mapped to the input image, and the contour of the foreground object in the input image is obtained. Of course, optionally, the positions of each pixel with a grayscale value of 1 in the first mask image can also be determined, and these pixel positions are located in the input image, wherein these pixel positions in the input image are the pixel positions occupied by the foreground object in the input image, and then, in combination with these pixel positions, the edge contour corresponding to the foreground object is determined from the input image.

[0094] In order to more intuitively understand the difference between the input image and the first mask image, as well as the above process of determining the contour of the foreground object in the input image according to the first mask image, Figure 2 To illustrate by way of example.

[0095] exist Figure 2 In the above, it is assumed that the input image is the image A1 shown in the figure, and the first mask image is the mask image A2 shown in the figure. In the process of generating the mask image A2, it is recognized that the image A1 contains a foreground object, the dress, so that the pixels corresponding to the foreground object in the mask image A2 are rendered black, and the other pixels are rendered white. At this time, the pixels on the boundary line of the black area are the pixels that constitute the outline of the foreground object. The positions of the pixels contained in the boundary line of the black area in the image A1 are located, so that the outline corresponding to the foreground object in the image A1 can be obtained.

[0096] Figure 2 The example in the figure shows the case where the input image contains a foreground object. Figure 3 To illustrate what the first mask image generated at this time will look like if the input image does not contain a foreground object.

[0097] like Figure 3 As shown, it is assumed that the input image is the image B1 shown in the figure, and the first mask image is the mask image B2 shown in the figure. In the process of generating the mask image B2, it is recognized that the image B1 does not contain a foreground object. Figure 3 As shown in , the mask image B2 will be a blank image, that is, all pixels therein are grayscale value 2, which is white. At this time, according to the mask image B2, it is found that there is no foreground object in the image B1, so the processing of the image B1 ends and the execution of the subsequent interaction process is not triggered.

[0098] As can be seen from the above description, in the process of generating the first mask image, it is necessary to identify whether there is a foreground object in the input image.

[0099] In an optional embodiment, the input image may be input into a semantic segmentation model to generate a first mask image through the semantic segmentation model. At this time, the semantic segmentation model may identify whether the input image contains a foreground object.

[0100] That is to say, a semantic segmentation model can be pre-trained so that the semantic segmentation model has the ability to identify whether there is a foreground object in the input image and the ability to generate a mask image corresponding to the input image.

[0101] The semantic segmentation model can be specifically implemented as any of the following network models: Fully Convolution Networks (FCN), SegNet, DeepLab, ENet, ICNet, etc.

[0102] Optionally, when the semantic segmentation model needs to be deployed in terminal devices with limited processing power and storage resources such as mobile terminals, the following semantic segmentation model can also be used: a semantic segmentation model composed of a MoGa (Mobile GPU-Aware, MoGa for short) network model as an encoder and an LR-ASPP (Lite reduce Atrous Spatial PyramidPooling, LR-ASPP for short) segmentation head network model as a decoder.

[0103] Combine the following Figure 4 The main working principle of the semantic segmentation model composed of the MoGa network model and the LR-ASPP segmentation head network model is briefly illustrated.

[0104] The input image is represented as image X. The MoGa network model contains multiple convolutional layers. Through the processing of each convolutional layer, semantic vectors of different dimensions can be obtained. For example Figure 4 As shown in the figure, assuming that the dimension of image X is S, the processing of four convolutional layers obtains semantic vectors of S / 2, S / 4, S / 8, and S / 16 respectively. Figure 4 In the figure, the semantic vectors of these four dimensions are represented as: semantic vector a, semantic vector b, semantic vector c, and semantic vector d. It can be considered that after each layer of convolutional layer processing, some semantics of background objects will be filtered out. Thus, through the processing of multiple layers of convolutional layers, the semantics of foreground objects can be gradually extracted.

[0105] The semantic vector d of S / 16 output by the MoGa network model is input into the LR-ASPP segmentation head network model. First, the LR-ASPP segmentation head network model extracts a higher-level semantic vector from the semantic vector d of S / 16. Figure 4 In the example, the semantic vector e is represented as a semantic vector e, and then the semantic vector e is upsampled to generate a semantic vector f of dimension S / 16. The semantic vector e can be understood as supervisory information, which contains more accurate semantics of foreground objects. Then, the LR-ASPP segmentation head network model multiplies the semantic vector d of S / 16 and the semantic vector f to obtain a semantic vector g of S / 16. Compared with the semantic vector d, due to the above supervisory information, the semantic vector g will contain richer semantics of foreground objects. After that, the semantic vector g is upsampled to generate a semantic vector h of dimension S / 8, and then the semantic vector h and the semantic vector c of S / 8 are superimposed to obtain the semantic vector i. The reason for superimposing the semantic vector c is that the semantic vector c of S / 8 contains the edge (i.e., contour) semantic information of the foreground object compared to the semantic vector d of S / 16. In order to ensure the accuracy of the contour recognition result of the foreground object, the semantic information of the semantic vector c of S / 8 is superimposed. Finally, based on the superposition result, a mask image Y corresponding to the image X can be obtained. The dimension of the mask image Y is the same as that of the image X, which is S.

[0106] The above describes the process of identifying whether an input image contains a foreground object and generating a first mask image based on a semantic segmentation model. The training process of the semantic segmentation model will be described below.

[0107] In another optional embodiment, it is also possible to identify whether the input image contains a foreground object and generate a first mask image in the following manner: obtain the depth information of each pixel in the input image, divide a plurality of pixel groups according to the depth information of each pixel, wherein the depth information of the pixels in each pixel group is the same or the difference is less than a set threshold; filter out the target pixel group with the smallest depth information; if the difference between the depth information corresponding to the target pixel group and the depth information corresponding to the other pixel groups is greater than the set threshold, and the size corresponding to the target pixel group is greater than the size corresponding to the other pixel groups, it is determined that the input image contains a foreground object. Then, according to the position of the pixels contained in the target pixel group, generate a first mask image, in which the pixel positions corresponding to the pixels contained in the target pixel group are rendered as grayscale value 1, and the other pixel positions are rendered as grayscale value 2.

[0108] After determining the corresponding contour of the foreground object in the input image according to the gray value difference between the foreground object and the non-foreground object in the first mask image, the input image may be interactively processed according to the contour corresponding to the foreground object. Alternatively, the contour may be displayed.

[0109] In general, the interactive processing can be to directly change the display effect of the input image, or to display some interactive elements for the user to use, so that the user can change the display effect of the input image according to his own needs. Figure 5 , Figure 7 , Figure 8 To provide several optional interaction methods by way of example.

[0110] Figure 5 A schematic diagram of interface changes of an image interaction method provided by an embodiment of the present invention, in Figure 5 In the example, it is assumed that the input image is the image A1 in the example above, and the foreground object is a dress. At this time, the interactive processing of the image A1 can be: displaying a set image within the contour corresponding to the foreground object, such as Figure 5 In practical applications, the image Z1 can be used to prompt the user which object in the image A1 should be focused on.

[0111] In addition, in some application scenarios, optionally, attribute information of the foreground object may also be obtained, so that the attribute information is displayed in association with the set image.

[0112] For ease of understanding, let's take the online shopping scenario as an example. Figure 6 To illustrate by way of example.

[0113] Assume that the user is currently browsing the above-mentioned image A1 in a certain product interface. Generate a mask image A2 corresponding to the image A1. Based on the mask image A2, it can be known that the image A1 contains a foreground object - a dress. Then, based on the mask image A2, the corresponding outline of the dress in the image A1 can be determined. Since the foreground object is the dress, in order to allow the user to clearly understand the attribute information of the dress such as price, brand, thickness, etc. when viewing this image, the recognition result based on the outline of the dress can be displayed within the outline range of the dress. Figure 6 The image Z1 shown in the figure is used to let the user know that the user should focus on the dress, and the attribute information of the dress, such as the price - 150 yuan, the brand - ANTP, etc., is displayed in association with the image Z1. In this way, the user does not need to check other locations in the interface that introduce the above attribute information to understand this information.

[0114] Figure 7A schematic diagram of interface changes of another image interaction method provided by an embodiment of the present invention, in Figure 7 In the example, it is assumed that the input image is the image A1 in the example above, and the foreground object is a dress. Optionally, the interactive processing of the image A1 may be: in response to the background replacement operation triggered on the image A1, the non-foreground object (i.e., the background object) in the image A1 is updated. In this interactive mode, a control for triggering the background replacement operation may be displayed in association with the image A1, such as Figure 7 The button of changing background 701 shown in FIG. Optionally, some optional background materials can also be provided for selection.

[0115] Still taking the online shopping scenario as an example, since the original background in image A1 may give the foreground object a certain visual effect, when shopping, the user wants to see the visual effect of the foreground object under background patterns of different styles and brightness, which can be achieved through the above-mentioned interactive method of changing the background object.

[0116] Figure 8 A schematic diagram of interface changes of another image interaction method provided by an embodiment of the present invention, in Figure 8 In the example, it is assumed that the input image is the image A1 in the above example, and the foreground object is a dress. Optionally, the interactive processing of the image A1 may be: obtaining the attribute information of the foreground object, determining the image of the target object corresponding to the attribute information, and displaying the image of the target object in association with the foreground object, wherein the target object and the foreground object have a matching relationship. Optionally, the attribute information includes at least one of the following: category information, color information.

[0117] For example, Figure 8 As shown in , it is assumed that the category information of the dress as the foreground object in image A1 is dress and the color information is white. In addition, it is assumed that the preset multiple objects that have a matching relationship with the dress include: sun hat, scarf, bag. Then Figure 8 As shown in , the images corresponding to these various objects can be displayed in association with the image A1, such as Figure 8 The images Z2, Z3, Z4, and Z5 shown in the figure may be images of some commodities selected from the database. Then, according to the user's selection operation on the images of these multiple objects, the image of the selected target object may be determined, assuming it is image Z2, so that the image Z2 of the target object may be displayed in association with the foreground object - the dress. Figure 8 In the figure, the associated display refers to wearing the sun hat indicated in the image Z2 on the head of the model in the image A1.

[0118] In addition to the image processing method provided above, optionally, in another embodiment, this document also provides the following image processing method, which may specifically include the following steps:

[0119] Acquire an image, wherein the image contains a foreground object;

[0120] generating a mask image corresponding to the image;

[0121] The pixels corresponding to the foreground object in the mask image are set to a first grayscale value range, and the pixels corresponding to the non-foreground object in the mask image are set to a second grayscale value range;

[0122] According to the gray value difference between the foreground object and the non-foreground object, the foreground object is cut out from the image.

[0123] Based on the image processing method, image cutout processing can be achieved, specifically cutting out the foreground object contained in the image.

[0124] Among them, according to the gray value difference between the foreground object and the non-foreground object, the foreground object is cut out in the image. Specifically, it can be: according to the gray value difference between the foreground object and the non-foreground object, the contour corresponding to the foreground object is first determined in the image, and then the foreground object is cut out from the image based on the contour.

[0125] In the above-mentioned image processing method, the generation of the mask image and the process of determining the contour of the foreground object can refer to the relevant descriptions in the above-mentioned other embodiments, which will not be repeated here.

[0126] In addition, optionally, in another embodiment, this document also provides the following image processing method, which may specifically include the following steps:

[0127] Acquire an image, wherein the image contains a foreground object;

[0128] generating a mask image corresponding to the image;

[0129] Setting pixels corresponding to foreground objects in the mask image to a first grayscale value range, and setting pixels corresponding to non-foreground objects in the mask image to a second grayscale value range;

[0130] According to the gray value difference between the foreground object and the non-foreground object, a first layer corresponding to the foreground object and a second layer corresponding to the non-foreground object are generated.

[0131] Based on the image processing method, layered processing of the image can be achieved, specifically, generating a first layer corresponding to the foreground object in the image and a second layer corresponding to the non-foreground object in the image.

[0132] Among them, according to the gray value difference between the foreground object and the non-foreground object, a first layer corresponding to the foreground object and a second layer corresponding to the non-foreground object are generated. Specifically, it can be: according to the gray value difference between the foreground object and the non-foreground object, the contour corresponding to the foreground object is first determined in the image, and then the foreground object is cut out from the image based on the contour to generate a first layer containing the foreground object, wherein the first layer is also an image, and its size can be equal to the size of the above-mentioned image containing the foreground object and the non-foreground object. The second layer can be an image obtained by cutting out the foreground object from the image. Based on the segmentation of the first layer and the second layer, the user can interact with the first layer and the second layer according to needs.

[0133] In the above-mentioned image processing method, the process of generating the mask image and the like can refer to the relevant description in the above-mentioned other embodiments, which will not be described in detail here.

[0134] In addition, optionally, in another embodiment, this document also provides the following commodity publishing method, which may specifically include the following steps:

[0135] Acquire a first image, wherein the first image includes a product to be released;

[0136] generating a mask image corresponding to the first image;

[0137] The pixels corresponding to the product in the mask image are set to a first grayscale value range, and the pixels corresponding to the background object in the first image in the mask image are set to a second grayscale value range;

[0138] Determine the contour corresponding to the product in the first image according to the gray value difference between the product and the background object;

[0139] A second image corresponding to the product is generated according to the outline of the product in the first image and the product publishing template, so as to publish the product through the second image.

[0140] In order to facilitate merchants to publish product images with good visual effects, it is assumed that the online shopping platform provides various product release templates for merchants to choose from. For example, for a product such as a women's top, a product release template is a video or image of a model wearing the product and walking on the street. In addition, in this embodiment, it is assumed that the product that a merchant needs to release is a sweater. When releasing the sweater, the merchant can take an image containing the sweater as the above-mentioned first image, such as taking an image of the sweater spread on the table. Based on this, the foreground object in the above-mentioned mask image is the sweater, and the background object is the table.

[0141] The above-mentioned method of generating a second image corresponding to the product based on the outline corresponding to the product in the first image and the product release template can be specifically as follows: according to the outline corresponding to the product in the first image, an image area corresponding to the product is cut out from the first image, and then, assuming that the product release template selected by the merchant is the product release template exemplified above, the cut-out image area can be appropriately transformed to add the transformed image to the corresponding position on the product release template, thereby obtaining the above-mentioned second image to be released, and displaying the second image in the product display interface.

[0142] The following describes the training process of the semantic segmentation model mentioned above.

[0143] Fig. 9 A schematic diagram of a training process of a semantic segmentation model provided by an embodiment of the present invention is shown in FIG. Fig. 9 As shown, the training process may include the following steps:

[0144] 901. Obtain a sample image.

[0145] 902. Identify the contours of each object contained in the sample image.

[0146] 903. Determine the outline of the selected object according to the selection operation on each object, where the selected object is a foreground object in the sample image.

[0147] 904. Generate a second mask image corresponding to the sample image according to the contour of the selected object.

[0148] 905. Under the supervision of the second mask image, the semantic segmentation model is trained according to the sample image.

[0149] When training a semantic segmentation model, a large number of training samples need to be collected, including positive sample images and negative sample images, where the positive sample images are images containing foreground objects and the negative sample images are images that do not contain foreground objects.

[0150] In this embodiment, the semantic segmentation model is trained in a supervised training manner, wherein the mask image of each sample image as a training sample is used as the supervisory information.

[0151] It can be understood that, for the negative sample image, since there is no foreground object in it, the corresponding mask image is an image with all pixels having a gray value of 2, that is, a white image. For the positive sample image, in its corresponding mask image, the pixels corresponding to the foreground object are set to a gray value of 1 (black), and the pixels corresponding to other non-foreground objects are set to a gray value of 2 (white).

[0152] It can be seen that before training, it is necessary to generate a corresponding mask image as supervision information for the positive sample image.

[0153] Assuming that the above sample image is any one of the positive sample images, taking the sample image as an example, the process of generating the corresponding second mask image may be:

[0154] First, the contours of each object contained in the sample image can be identified by contour extraction algorithms such as OpenCV. That is to say, the sample image will contain multiple objects, including a foreground object and one or more background objects. At this time, for all objects contained in the sample image, their corresponding contours are extracted. Furthermore, the contour extraction results of each object can be displayed on the interface so that the person responsible for model training can see the contours of each object. Based on this, the person can select the foreground object (for example, trigger a click operation in the area surrounded by the contour corresponding to the foreground object). At this time, based on the contour of the selected foreground object, a second mask image corresponding to the sample image can be generated, that is, the pixels in the area surrounded by the contour are set to grayscale value 1, and the other pixels are set to grayscale value 2 to obtain the second mask image.

[0155] After obtaining the second mask image corresponding to the sample image, the sample image is input into the semantic segmentation model under the supervision of the second mask image to perform supervised training on the semantic segmentation model.

[0156] In another embodiment, in order to improve the model training efficiency and the diversity of training samples, the already collected training samples may be expanded.

[0157] Taking the above sample image as an example, the extended processing is embodied as follows: performing the same image enhancement processing on the sample image and the second mask image, and training the semantic segmentation model with the enhanced sample image and mask image.

[0158] Optionally, the above-mentioned image enhancement processing includes any of the following: random Gaussian blur, random flipping, random rotation, random cropping, random local deformation, random affine transformation, random motion blur, adding random noise, random color enhancement, random highlighting. Of course, it is not limited to this.

[0159] Combine the following Fig.10 and Fig.11 The effects of the above two image enhancement processing methods are shown.

[0160] exist Fig.10In the example, it is assumed that a sample image is image C1, and the foreground object in image C1 is the mouse shown in the figure. It is assumed that image C1 is transformed into image D1 through a certain flip transformation process. In image D1, the position of the mouse has changed compared with image C1.

[0161] It can be understood that the flip transformation process also acts on the mask image corresponding to the image C1, so that the mask image corresponding to the image D1 is obtained from the mask image corresponding to the image C1.

[0162] Based on this, another training sample, image D1, is expanded from the already collected training sample, image C1. It can be understood that more training samples can be expanded from image C1 by updating the specific flipping form of the flipping transformation.

[0163] exist Fig.11 In the example, it is still assumed that a sample image is image C1. It is assumed that image C1 is changed into image E1 by some cropping method. In image E1, the mouse part is cropped.

[0164] The cropping operation also acts on the mask image corresponding to the image C1, so that the mask image corresponding to the image E1 is obtained from the mask image corresponding to the image C1. Based on this, another training sample, image E1, is expanded from the already collected training sample, image C1. It can be understood that more training samples can be expanded from image C1 by updating the cropping position and the cropping window size.

[0165] The image processing apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these image processing apparatuses can be configured using commercially available hardware components through the steps taught in this solution.

[0166] Fig.12 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention is shown in FIG. Fig.12 As shown, the device includes: an acquisition module 11, a generation module 12, a setting module 13, and a determination module 14.

[0167] The acquisition module 11 is used to acquire an image, wherein the image contains a foreground object.

[0168] The generating module 12 is used to generate a first mask image corresponding to the image.

[0169] The setting module 13 is used to set the pixels corresponding to the foreground object in the first mask image to a first gray value range, and set the pixels corresponding to the non-foreground object in the first mask image to a second gray value range.

[0170] The determination module 14 is used to determine the contour corresponding to the foreground object in the image according to the gray value difference between the foreground object and the non-foreground object.

[0171] Optionally, the generating module 12 may be specifically configured to: input the image into a semantic segmentation model to generate the first mask image through the semantic segmentation model.

[0172] Optionally, the semantic segmentation model consists of a MoGa network model as an encoder and an LR-ASPP segmentation head network model as a decoder.

[0173] Optionally, the device also includes: a model training module.

[0174] The model training module is used to obtain a sample image and a second mask image corresponding to the sample image; and train the semantic segmentation model according to the sample image under the supervision of the second mask image.

[0175] Optionally, in the process of obtaining a second mask image corresponding to the sample image, the model training module is specifically used to: identify the contours of each object contained in the sample image; determine the contours of the selected objects based on the selection operation of the objects, the selected objects being the foreground objects in the sample image; and generate a second mask image corresponding to the sample image based on the contours of the selected objects.

[0176] Optionally, the model training module is further used to: perform the same image enhancement processing on the sample image and the second mask image, and the enhanced sample image and mask image are used for training the semantic segmentation model.

[0177] Optionally, the image enhancement processing includes any one of the following: random Gaussian blur, random flipping, random rotation, random cropping, random local deformation, random affine transformation, random motion blur, adding random noise, random color enhancement, and random highlighting.

[0178] Optionally, the device may further include: a display module, configured to display the outline corresponding to the foreground object.

[0179] Optionally, the device may further include: an interactive module, configured to interactively process the image according to a contour corresponding to the foreground object.

[0180] Optionally, the interaction module may be specifically configured to: display a set image within a contour corresponding to the foreground object.

[0181] Optionally, the interaction module may also be used to: obtain attribute information of the foreground object; and display the attribute information in association with the set image.

[0182] Optionally, the interaction module may also be configured to: update non-foreground objects in the first image in response to a background replacement operation triggered on the first image.

[0183] Optionally, the interaction module can also be used to: obtain attribute information of the foreground object; determine an image of a target object corresponding to the attribute information, the target object having a matching relationship with the foreground object; and display the image of the target object in association with the foreground object.

[0184] Optionally, the interaction module may also be used to: display images of multiple objects corresponding to the attribute information; and determine the image of the target object according to a selection operation on the images of the multiple objects.

[0185] Optionally, the attribute information includes at least one of the following: category information and color information.

[0186] Fig.12 The device shown can perform the aforementioned Figures 1 to 11 The image processing method provided in the illustrated embodiment, the detailed execution process and technical effects can be found in the description of the aforementioned embodiment, which will not be repeated here.

[0187] In one possible design, the above Fig.12 The structure of the image processing device shown can be implemented as an electronic device, such as Fig.13 As shown, the electronic device may include: a processor 21 and a memory 22. The memory 22 stores executable code, and when the executable code is executed by the processor 21, the processor 21 can at least implement the above-mentioned Figures 1 to 11 The image processing method provided in the illustrated embodiment.

[0188] Optionally, the electronic device may further include a communication interface 23 for communicating with other devices.

[0189] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, wherein an executable code is stored on the non-transitory machine-readable storage medium. When the executable code is executed by a processor of an electronic device, the processor can at least implement the above-mentioned Figures 1 to 11 The image processing method provided in the illustrated embodiment.

[0190] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0191] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0192] The image processing method provided in the embodiment of the present invention can be executed by a certain program / software, and the program / software can be provided by the network side. The electronic device mentioned in the above embodiment can download the program / software to a local non-volatile storage medium, and when it needs to execute the above image processing method, the program / software is read into the memory by the CPU, and then the CPU executes the program / software to implement the image processing method provided in the above embodiment. The execution process can refer to the above Figures 1 to 11 Instructions in .

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire an image and depth information of each pixel in the image, wherein the image contains a foreground object; Dividing a plurality of pixel groups according to the depth information of each pixel, wherein the depth information of the pixels in each pixel group is the same or the difference is less than a set threshold; Filtering out a target pixel group having the smallest depth information from the plurality of pixel groups; If the difference between the depth information corresponding to the target pixel group and the depth information corresponding to other pixel groups is greater than a set threshold, and the size corresponding to the target pixel group is greater than the size corresponding to other pixel groups, it is determined that the image contains a foreground object; generating a first mask image corresponding to the image according to positions of pixels included in the target pixel group; Setting pixels corresponding to the foreground object in the first mask image to a first grayscale value range, and setting pixels corresponding to non-foreground objects in the first mask image to a second grayscale value range; According to the difference in grayscale values ​​between the foreground object and the non-foreground object, a contour corresponding to the foreground object in the image is determined.

2. The method according to claim 1, characterized in that The method further comprises: The outline corresponding to the foreground object is displayed.

3. The method according to claim 1, characterized in that The method further comprises: The image is interactively processed according to the contour corresponding to the foreground object.

4. The method according to claim 3, characterized in that The interactive processing of the image according to the contour corresponding to the foreground object includes: A set image is displayed within the outline corresponding to the foreground object.

5. The method according to claim 4, characterized in that The method further comprises: Acquiring attribute information of the foreground object; The attribute information is displayed in association with the setting image.

6. The method according to claim 3, characterized in that The interactive processing of the image according to the contour corresponding to the foreground object includes: In response to a background replacement operation being triggered on a first image, non-foreground objects in the first image are updated.

7. The method according to claim 3, characterized in that The interactive processing of the image according to the contour corresponding to the foreground object includes: Acquiring attribute information of the foreground object; Determining an image of a target object corresponding to the attribute information, the target object having a matching relationship with the foreground object; The image of the target object is displayed in association with the foreground object.

8. The method according to claim 7, characterized in that The determining the image of the target object corresponding to the attribute information includes: displaying images of a plurality of objects corresponding to the attribute information; According to the selection operation of the images of the plurality of objects, the image of the target object is determined.

9. An image processing device, characterized in that: include: An acquisition module, used to acquire an image and depth information of each pixel in the image, wherein the image contains a foreground object; A generating module, configured to divide the image into a plurality of pixel groups according to the depth information of each pixel, wherein the depth information of the pixels in each pixel group is the same or the difference is less than a set threshold; select a target pixel group with the smallest depth information from the plurality of pixel groups; if the difference between the depth information corresponding to the target pixel group and the depth information corresponding to other pixel groups is greater than a set threshold, and the size corresponding to the target pixel group is greater than the size corresponding to other pixel groups, then determine that the image contains a foreground object; and generate a first mask image corresponding to the image according to the positions of the pixels contained in the target pixel group; A setting module, used to set the pixels corresponding to the foreground object in the first mask image to a first gray value range, and set the pixels corresponding to the non-foreground object in the first mask image to a second gray value range; The determination module is used to determine the contour corresponding to the foreground object in the image according to the gray value difference between the foreground object and the non-foreground object.

10. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the image processing method as claimed in any one of claims 1 to 8.

11. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the image processing method according to any one of claims 1 to 8.

12. An image processing method, characterized in that: include: Acquire an image and depth information of each pixel in the image, wherein the image contains a foreground object; Dividing a plurality of pixel groups according to the depth information of each pixel, wherein the depth information of the pixels in each pixel group is the same or the difference is less than a set threshold; Filtering out a target pixel group having the smallest depth information from the plurality of pixel groups; If the difference between the depth information corresponding to the target pixel group and the depth information corresponding to other pixel groups is greater than a set threshold, and the size corresponding to the target pixel group is greater than the size corresponding to other pixel groups, it is determined that the image contains a foreground object; generating a mask image corresponding to the image according to the positions of the pixels included in the target pixel group; Setting the pixels corresponding to the foreground object in the mask image to a first grayscale value range, and setting the pixels corresponding to the non-foreground object in the mask image to a second grayscale value range; The foreground object is cut out from the image according to a grayscale value difference between the foreground object and the non-foreground object.

13. An image processing method, characterized in that: include: Acquire an image and depth information of each pixel in the image, wherein the image contains a foreground object; Dividing a plurality of pixel groups according to the depth information of each pixel, wherein the depth information of the pixels in each pixel group is the same or the difference is less than a set threshold; Filtering out a target pixel group having the smallest depth information from the plurality of pixel groups; If the difference between the depth information corresponding to the target pixel group and the depth information corresponding to other pixel groups is greater than a set threshold, and the size corresponding to the target pixel group is greater than the size corresponding to other pixel groups, it is determined that the image contains a foreground object; generating a mask image corresponding to the image according to the positions of the pixels included in the target pixel group; Setting the pixels corresponding to the foreground object in the mask image to a first grayscale value range, and setting the pixels corresponding to the non-foreground object in the mask image to a second grayscale value range; According to the gray value difference between the foreground object and the non-foreground object, a first layer corresponding to the foreground object and a second layer corresponding to the non-foreground object are generated.

14. A method for releasing a product, characterized in that: include: Acquire a first image and depth information of each pixel in the first image, wherein the first image contains a product to be released; Dividing a plurality of pixel groups according to the depth information of each pixel, wherein the depth information of the pixels in each pixel group is the same or the difference is less than a set threshold; Filtering out a target pixel group having the smallest depth information from the plurality of pixel groups; If the difference between the depth information corresponding to the target pixel group and the depth information corresponding to other pixel groups is greater than a set threshold, and the size corresponding to the target pixel group is greater than the sizes corresponding to other pixel groups, it is determined that the first image contains a product to be released; generating a mask image corresponding to the first image according to positions of pixels included in the target pixel group; Set the pixels corresponding to the product in the mask image to a first grayscale value range, and set the pixels corresponding to the background objects in the first image in the mask image to a second grayscale value range; Determining a contour corresponding to the product in the first image according to a grayscale value difference between the product and the background object; A second image corresponding to the product is generated according to the outline of the product in the first image and a product publishing template, so as to publish the product through the second image.

Images