Image processing method and device, computer device and storage medium

Abstract

The application discloses an image processing method and device, computer equipment and a storage medium, comprising: collecting a target image to be processed; inputting the target image into a preset segmentation model; reading a feature mask image of the target selected area output by the segmentation model, and performing mask processing on the feature mask image and the target image to generate a target foreground image of the target image; and performing image enhancement processing on the target foreground image according to a preset image enhancement strategy to generate an extracted image of the target image. The edge color difference value of the feature mask image is more obvious, and the difference between the image of the target selected area and the background pixels can be more easily distinguished. After the target image is filtered by the feature mask image obtained by segmentation and extraction, the target foreground image obtained is more complete, and the final extracted image is more complete and clear.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing, and in particular to an image processing method, apparatus, computer device and storage medium. Background Technology

[0002] Image processing is the technique of using computers to analyze images to achieve desired results. It is also known as image processing. Image processing generally refers to digital image processing.

[0003] The inventors of this invention discovered during their research that product backgrounds cannot be complex when placing e-commerce advertisements. Therefore, they used image cutout technology to retain only the product and remove the background, replacing it with a solid color. When using templates for image creation, cutout images are often required. Images with backgrounds reduce the image quality, thus affecting store design and advertising effectiveness. Using cutout images allows for easy template application, quickly creating materials, lowering the barrier to entry for material creation, and improving work efficiency. Existing technologies for image cutout... Figure 1 Manual image cutout is commonly used, but it has a high skill ceiling, requiring specialized techniques, and is relatively inefficient. With technological advancements, many methods using deep learning for semantic and entity segmentation have emerged, but these methods suffer from unclear and incomplete cutouts when applied to product image cutout. Summary of the Invention

[0004] This invention provides an image processing method, apparatus, computer device, and storage medium that can improve image matting efficiency and increase the integrity of image matting.

[0005] To solve the above-mentioned technical problems, one technical solution adopted by the embodiments of the present invention is: to provide an image processing method, comprising:

[0006] Acquire a target image to be processed, wherein the target image includes a target selection area that represents a preset target object image;

[0007] The target image is input into a preset segmentation model, wherein the segmentation model is a neural network model pre-trained to a convergent state for extracting and segmenting images.

[0008] Read the feature mask image of the target selection region output by the segmentation model, and perform mask processing on the feature mask image and the target image to generate the target foreground image of the target image;

[0009] According to a preset image enhancement strategy, the target foreground image is subjected to image enhancement processing to generate an extracted image of the target image.

[0010] Optionally, the step of performing masking processing on the feature mask image and the target image to generate the target foreground image of the target image includes:

[0011] Generate a mask matrix for the target image based on the feature mask image;

[0012] The mask matrix is ​​multiplied by the image matrix of the target image to generate the foreground matrix;

[0013] The foreground matrix is ​​pixelated to generate the target foreground image.

[0014] Optionally, the image enhancement strategy includes: performing noise removal processing on the target foreground image; the step of performing image enhancement processing on the target foreground image according to a preset image enhancement strategy to generate an extracted image of the target image includes:

[0015] Fill the background region of the target foreground image with preset background pixels to generate the first foreground image;

[0016] The first foreground image is binarized according to a preset first pixel threshold to generate a binary image of the first foreground image.

[0017] The binary image is processed by connected component processing, which connects pixels with the same pixel value in the binary image to generate multiple region blocks.

[0018] The extracted image is generated by filtering the area of ​​the multiple region blocks.

[0019] Optionally, the step of performing area filtering on the plurality of region blocks to generate the extracted image includes:

[0020] Extract the region block with the largest area from the plurality of region blocks as the reference region block;

[0021] The area of ​​each of the multiple region blocks is proportionally calculated with the reference region block to generate the area ratio of each region block to the reference region block;

[0022] The regions whose area ratio is less than a preset ratio threshold or whose area is less than a preset area threshold are filtered out, and the filtered regions are filled with background pixels to generate the extracted image.

[0023] Optionally, the image enhancement strategy includes: enhancing the sharpness of the target foreground image, wherein performing image enhancement processing on the target foreground image according to a preset image enhancement strategy to generate an extracted image of the target image includes:

[0024] Read the pixels in the feature mask image sequentially;

[0025] The image selection area is generated based on the read pixels and a preset radius value.

[0026] Read the pixel values ​​in the selected area of ​​the image and determine whether at least one pixel value in the selected area of ​​the image is equal to a preset second pixel threshold;

[0027] If at least one pixel value in the selected area of ​​the image is equal to a preset second pixel threshold, the pixel is a pixel near the boundary and its value remains unchanged; otherwise, the pixel value of the read pixel is filled with a preset fill pixel value.

[0028] The extracted image is generated by outlining the target selection area in the target foreground image based on the read pixels.

[0029] Optionally, the image enhancement strategy includes: performing thin band restoration processing on the target foreground image; the step of performing image enhancement processing on the target foreground image according to a preset image enhancement strategy to generate an extracted image of the target image includes:

[0030] The target image is input into a preset thin band detection model, wherein the thin band detection model is a neural network model pre-trained to a convergent state for detecting images;

[0031] The detected image output by the thin band detection model is input into a preset thin band image segmentation model, wherein the thin band image segmentation model is a neural network model pre-trained to a convergent state for thinning and segmenting the image.

[0032] The thin band image output by the thin band image segmentation model is read, and the thin band image is fused with the target foreground image to generate the extracted image.

[0033] Optionally, the thin-strip image segmentation model includes a u2net model and a Sobel operator. The u2net model includes four U-block modules, each of which includes a downsampling channel and an upsampling channel. The downsampling channel includes two 3*3 convolutional layers and multiple 2*2 convolutional layers. The input of each of the multiple 2*2 convolutional layers includes the outputs of the two convolutional layers above it. The thin-strip image is a fused image of the segmented images from the u2net model and the Sobel operator.

[0034] To address the aforementioned technical problems, embodiments of the present invention also provide an image processing apparatus, comprising:

[0035] The acquisition module is used to acquire the target image to be processed, wherein the target image includes a target selection area that represents a preset target object image;

[0036] The segmentation module is used to input the target image into a preset segmentation model, wherein the segmentation model is a neural network model pre-trained to a convergent state for extracting and segmenting images.

[0037] The processing module is used to read the feature mask image of the target selection area output by the segmentation model, and perform mask processing on the feature mask image and the target image to generate the target foreground image of the target image;

[0038] The enhancement module is used to perform image enhancement processing on the target foreground image according to a preset image enhancement strategy, and generate an extracted image of the target image.

[0039] Optionally, the image processing apparatus further includes:

[0040] The first mask submodule is used to generate a mask matrix of the target image based on the feature mask image;

[0041] The first operation submodule is used to perform a multiplication operation between the mask matrix and the image matrix of the target image to generate a foreground matrix;

[0042] The first processing submodule is used to pixelate the foreground matrix to generate the target foreground image.

[0043] Optionally, the image enhancement strategy includes: performing noise removal processing on the target foreground image, and the image processing device further includes:

[0044] The first filling submodule is used to fill the background area of ​​the target foreground image with preset background pixels to generate the first foreground image;

[0045] The second processing submodule is used to perform binarization processing on the first foreground image according to a preset first pixel threshold to generate a binary image of the first foreground image.

[0046] The first connected component submodule is used to perform connected component processing on the binary image, so that pixels with the same pixel value in the binary image are connected to form multiple region blocks.

[0047] The first generation submodule is used to generate the extracted image by filtering the area of ​​the multiple region blocks.

[0048] Optionally, the image processing apparatus further includes:

[0049] The first extraction submodule is used to extract the region block with the largest area among the multiple region blocks as the reference region block;

[0050] The second calculation submodule is used to perform proportional calculations between the area of ​​each of the multiple region blocks and the reference region block to generate the area ratio of each region block to the reference region block.

[0051] The first filtering submodule is used to filter out regions whose area ratio is less than a preset ratio threshold or whose area is less than a preset area threshold, and fill the filtered regions with background pixels to generate the extracted image.

[0052] Optionally, the image enhancement strategy includes: enhancing the sharpness of the target foreground image, and the image processing device further includes:

[0053] The first reading submodule is used to sequentially read the pixels in the feature mask image;

[0054] The second generation submodule is used to generate an image selection area based on a preset radius value, with the read pixels as the center.

[0055] The first confirmation submodule is used to read the pixel values ​​in the image selection area and determine whether at least one pixel value in the image selection area is equal to a preset second pixel threshold.

[0056] The second filling submodule is used to fill the pixel value of the read pixel point with a preset filling pixel value when at least one pixel value in the selected area of ​​the image is equal to a preset second pixel threshold, and the pixel point is a pixel near the boundary.

[0057] The third generation submodule is used to outline the target selection area in the target foreground image based on the read pixels, and generate the extracted image.

[0058] Optionally, the image enhancement strategy includes: performing thin band restoration processing on the target foreground image, and the image processing device further includes:

[0059] The third processing submodule is used to input the target image into a preset thin band detection model, wherein the thin band detection model is a neural network model pre-trained to a convergent state for detecting images;

[0060] The fourth processing submodule is used to input the detection image output by the thin band detection model into a preset thin band image segmentation model, wherein the thin band image segmentation model is a neural network model pre-trained to a convergent state for thinning and segmenting the image.

[0061] The fourth generation submodule is used to read the thin strip image output by the thin strip image segmentation model, and perform image fusion processing on the thin strip image and the target foreground image to generate the extracted image.

[0062] Optionally, the thin-strip image segmentation model includes a u2net model and a Sobel operator. The u2net model includes four U-block modules, each of which includes a downsampling channel and an upsampling channel. The downsampling channel includes two 3*3 convolutional layers and multiple 2*2 convolutional layers. The input of each of the multiple 2*2 convolutional layers includes the outputs of the two convolutional layers above it. The thin-strip image is a fused image of the segmented images from the u2net model and the Sobel operator.

[0063] To address the aforementioned technical problems, this invention also provides a computer device, including a memory and a processor. The memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the image processing method described above.

[0064] To address the aforementioned technical problems, this embodiment of the invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the image processing method described above.

[0065] The beneficial effects of this invention are as follows: By performing image segmentation on the target image to generate a feature mask image of the target object, and using the feature mask image as a mask for the target image, image filtering and selection are performed on the target image to extract a target foreground image representing the target object image. Furthermore, image enhancement is applied to the target foreground image to generate an extracted image of the target selection area. Because the edge color difference values ​​of the feature mask image are more obvious, it is easier to distinguish the difference between the target selection area image and the background pixels. After using the feature mask image obtained through segmentation and extraction to perform mask filtering on the target image, the resulting target foreground image is more complete, making the final extracted image more complete and clear. Attached Figure Description

[0066] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0067] Figure 1 This is a schematic diagram of the basic flow of an image processing method according to a specific embodiment of this application;

[0068] Figure 2 This is a schematic diagram of the target foreground image extraction process according to a specific embodiment of this application;

[0069] Figure 3 This is a schematic diagram illustrating the process of noise removal from a target foreground image according to a specific embodiment of this application.

[0070] Figure 4 This is a flowchart illustrating the filtering region block of a specific embodiment of this application;

[0071] Figure 5 This is a schematic diagram of the process for enhancing the sharpness of a target foreground image according to a specific embodiment of this application;

[0072] Figure 6 This is a schematic flowchart illustrating the thin band restoration process for a target foreground image according to a specific embodiment of this application.

[0073] Figure 7 This is a flowchart of the residual U-shaped unit (RSU) according to a specific embodiment of this application;

[0074] Figure 8 This is a schematic diagram of the basic structure of an image processing apparatus according to an embodiment of this application;

[0075] Figure 9 This is a basic structural block diagram of a computer device according to an embodiment of this application. Detailed Implementation

[0076] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0077] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the word “comprising” as used in the specification of this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0078] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0079] Those skilled in the art will understand that the term "terminal" as used herein includes both devices that are wireless signal receivers, devices that are wireless signal receivers without transmitting capability, and devices with receiving and transmitting hardware, having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such devices may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptop and / or handheld computers or other devices that have and / or include a radio frequency receiver. As used herein, "terminal" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally, and / or in a distributed manner, operating in any other location on Earth and / or in space. "Terminal" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0080] Please see Figure 1 , Figure 1 This is a schematic diagram of the basic process of the image processing method in this embodiment.

[0081] like Figure 1 As shown, an image processing method includes:

[0082] S1100. Acquire the target image to be processed, wherein the target image includes a target selection area that represents a preset target object image;

[0083] In this embodiment, the target image refers to a clothing image. However, the image content of the target image is not limited to this. Depending on the specific application scenario, in some embodiments, the image content of the target image may also be (not limited to): a person image, a face image, an object image, or an animal image.

[0084] The target image can be acquired in the following ways: images stored in local storage space, images obtained from the server, or images captured in real time by a camera.

[0085] S1200. Input the target image into a preset segmentation model, wherein the segmentation model is a neural network model pre-trained to a convergent state for extracting and segmenting images.

[0086] The acquired target image is input into a segmentation model, which is a pre-trained, converged neural network model used for image segmentation. The segmentation model is trained to convert the target image into a feature mask image and extract the target image region from the feature mask image. The target image region refers to the area occupied by the features of clothing or other objects in the scene that need to be extracted. The output of the segmented image is the feature mask image of the clothing or other objects in the target image.

[0087] The segmentation model in this embodiment includes (but is not limited to): a convolutional neural network model, a deep convolutional neural network model, a recurrent neural network model, or a variant of the above three models. For example, in some embodiments, the segmentation model adopts U... 2 .net model.

[0088] When training the segmentation model, binary feature mask images are used for the training set, and grayscale feature mask images are used for the output images. Training the segmentation images through supervised or semi-supervised training allows the segmentation images trained to convergence to capture more subtle color value differences, making the segmentation images more sensitive.

[0089] S1300: Read the feature mask image of the target selection area output by the segmentation model, and perform mask processing on the feature mask image and the target image to generate the target foreground image of the target image;

[0090] The segmentation model outputs a feature mask image of the target region extracted from the target model. The extracted feature mask image is a grayscale image. The feature mask image is composed of individual pixels, each with a pixel value. The pixel values ​​of each pixel in the feature mask image are extracted to generate the pixel matrix of the feature mask image. Since the feature mask image is obtained through segmentation, the pixels at the positions of background pixels are set to 0. Therefore, the resulting pixel matrix is ​​the mask matrix of the target image.

[0091] After extracting the mask matrix of the target image, the image matrix of the target image is extracted. Similarly, the target image is also composed of pixels, each with a corresponding pixel value. The image matrix of the target image is generated by reading the pixel values ​​of each pixel in the target image. Then, the mask matrix and the image matrix are multiplied. During the multiplication operation, since the pixels in the mask matrix at the corresponding positions in the background image are all 0, the multiplication operation can set the pixels at the corresponding positions in the image matrix to zero, thereby filtering out the target selection areas in the target image that are the same as those in the feature mask image, thus completing the extraction of clothing or other target object image regions in the target image.

[0092] The matrix generated by multiplying the mask matrix and the image matrix is ​​the foreground matrix. After generating the foreground matrix, the foreground matrix is ​​pixelated according to the correspondence between each pixel value and color in the foreground matrix. The image generated by the pixelation process is the target foreground image, which is the extracted image of clothing or other target objects.

[0093] In this embodiment, the feature mask image is a grayscale feature mask image. However, the color value range of the feature mask image is not limited to this. In some embodiments, the feature mask image is a binary feature mask image.

[0094] In this embodiment, the target foreground image is the extracted image of the target selection area obtained from the target image.

[0095] S1400. According to the preset image enhancement strategy, perform image enhancement processing on the target foreground image to generate the extracted image of the target image.

[0096] After generating the target foreground image, the target foreground image is enhanced according to a preset image enhancement strategy. The image generated after image enhancement is an extracted image of clothing or other target objects.

[0097] The image enhancement strategy in this embodiment can be (but is not limited to): any one of the following: noise removal processing of the target foreground image, sharpness enhancement processing of the target foreground image, or thin band restoration processing of the target foreground image. It can also be a processing strategy that performs noise removal processing of the target foreground image, sharpness enhancement processing of the target foreground image, and thin band restoration processing of the target foreground image in sequence.

[0098] In this embodiment, a feature mask image of the target object is generated by image segmentation of the target image. This feature mask image is then used as a mask for image filtering and selection of the target image to extract a target foreground image representing the target object. This foreground image is then enhanced to generate an extracted image of the target selection area. Because the edge color difference values ​​of the feature mask image are more pronounced, it is easier to distinguish between the target selection area and the background pixels. After using the segmented and extracted feature mask image for mask filtering of the target image, the resulting target foreground image is more complete, leading to a more complete and clear final extracted image.

[0099] In some implementations, a feature mask image is used as a mask to mask the target image and extract the target foreground image. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the target foreground image extraction process in this embodiment.

[0100] like Figure 2 As shown, S1300 includes:

[0101] S1311. Generate a mask matrix for the target image based on the feature mask image;

[0102] The segmentation model outputs a feature mask image of the target region extracted from the target model. The extracted feature mask image is a grayscale image. The feature mask image is composed of individual pixels, each with a pixel value. The pixel values ​​of each pixel in the feature mask image are extracted to generate the pixel matrix of the feature mask image. Since the feature mask image is obtained through segmentation, the pixels at the positions of background pixels are set to 0. Therefore, the resulting pixel matrix is ​​the mask matrix of the target image.

[0103] S1312. Multiply the mask matrix with the image matrix of the target image to generate a foreground matrix;

[0104] After extracting the mask matrix of the target image, the image matrix of the target image is extracted. Similarly, the target image is also composed of pixels, each with a corresponding pixel value. The image matrix of the target image is generated by reading the pixel values ​​of each pixel in the target image. Then, the mask matrix and the image matrix are multiplied. During the multiplication operation, since the pixels in the mask matrix at the corresponding positions in the background image are all 0, the multiplication operation can set the pixels at the corresponding positions in the image matrix to zero, thereby filtering out the target selection areas in the target image that are the same as those in the feature mask image, thus completing the extraction of clothing or other target object image regions in the target image.

[0105] S1313. The foreground matrix is ​​pixelated to generate the target foreground image.

[0106] The matrix generated by multiplying the mask matrix and the image matrix is ​​the foreground matrix. After generating the foreground matrix, the foreground matrix is ​​pixelated according to the correspondence between each pixel value and color in the foreground matrix. The image generated by the pixelation process is the target foreground image, which is the extracted image of clothing or other target objects.

[0107] In some implementations, the image enhancement strategy can be to remove noise from the target foreground image. See also... Figure 3 , Figure 3 This is a schematic diagram of the noise removal process for the target foreground image in this embodiment.

[0108] like Figure 3 As shown, S1400 includes:

[0109] S1411. Fill the background area of ​​the target foreground image with preset background pixels to generate a first foreground image;

[0110] The target foreground image has the same screen size as the target image; therefore, in addition to the target selection area, the target foreground image also contains a background area.

[0111] When denoising the target foreground image, it is necessary to fill the background area with pixels. The filling pixels are solid color pixels. For example, in some implementations, the filling background pixels are white. The method for filling the pure white background is: pure white background value = foreground pixel value + 255 * (1 - feature mask image pixel value). The background area of ​​the target foreground image is filled with pure white using the above formula.

[0112] It should be noted that the fill color of the background area is not limited to this. Depending on the specific implementation, in some implementations, the background pixels can be (but are not limited to) black, gray, green, or other colors required by the scene.

[0113] The target foreground image after the background area has been filled with background pixels is the first foreground image.

[0114] S1412. The first foreground image is binarized according to a preset first pixel threshold to generate a binary image of the first foreground image.

[0115] After generating the first foreground image, it needs to be binarized. The binarization process involves classifying the pixel values ​​of the pixels in the first foreground image into two colors: white and black, based on a first pixel threshold. For example, the pixel values ​​of pixels in the first foreground image that are greater than or equal to the first pixel threshold are changed to 255, while the pixel values ​​of pixels in the first foreground image that are less than the first pixel threshold are changed to 0. This completes the binarization operation of the first foreground image.

[0116] In some implementations, the first pixel threshold is set to 250, which means that the pixel values ​​of pixels in the first foreground image with a value greater than or equal to 250 are modified to 255, while the pixel values ​​of pixels in the first foreground image with a value less than 250 are modified to 0. The advantage of this setting is that it preserves as many elements in the target foreground image as possible after binarization, while masking out foreground elements that are close to white to avoid them being removed as noise, thus reducing information loss.

[0117] However, the value of the first pixel threshold is not limited to this. Depending on the specific application scenario, the user can set it according to actual needs, and it can be set to any value between 0 and 255 pixels. After binarization processing, a binary image of the first foreground image is generated.

[0118] S1413. Perform connected component processing on the binary image to connect pixels with the same pixel value in the binary image to generate multiple region blocks.

[0119] After generating a binary image, to ensure continuity between image regions, connected component processing is performed. This processing connects adjacent identical pixels, forming regions of the same color. Around each region, the colors of the surrounding pixels differ from those within the region. Since identical pixels in a binary image are sometimes not continuous, multiple regions are generated.

[0120] The number of blocks can be (but is not limited to): 2, 3, 4, 5 or more.

[0121] S1414. The area of ​​the multiple region blocks is filtered to generate the extracted image.

[0122] After generating multiple region blocks, the pixel area of ​​each region block is statistically analyzed. Then, based on the statistically obtained area, the multiple region blocks are filtered. The filtering methods can be (but are not limited to): deleting the top n region blocks with the smallest area, where n is a user-defined integer, and Top n refers to the top n smallest regions in the sorting; deleting region blocks with an area smaller than a set area threshold; deleting region blocks whose area ratio to the largest region block is less than a set ratio threshold; deleting region blocks with the smallest area but insufficient contour smoothness, etc.

[0123] After deleting regions that do not meet the filtering rules from multiple regions, the corresponding pixel blocks in the target foreground image are filled with background pixels, and the resulting image is the extracted image. By deleting noise pixels with excessively small areas in the target foreground image, the extracted image becomes clearer and more complete.

[0124] In some implementations, filtering of region blocks requires using area ratios or area thresholds. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating the process of filtering region blocks in this embodiment.

[0125] like Figure 4 As shown, S1414 includes:

[0126] S1421. Extract the region block with the largest area from the plurality of region blocks as the reference region block;

[0127] After generating multiple region blocks through connected component processing, the area of ​​each region block is counted based on the number of pixels it contains. After obtaining the area of ​​each region block, the areas of each region block are sorted in descending or ascending order to obtain the region block with the largest area, and this region block is determined as the reference region block.

[0128] S1422. Calculate the area of ​​each of the multiple region blocks by proportionally calculating the area of ​​the reference region block to generate the area ratio of each region block to the reference region block.

[0129] After obtaining the area of ​​each region block and the area of ​​the largest region block, the area ratio of each region block to the reference region block is calculated. This ratio is calculated as a percentage between 0 and 1. The area ratio of each region block to the reference region block is then associated with its corresponding region block to generate the area ratio for each region block.

[0130] S1423. Filter out the regions whose area ratio is less than a preset ratio threshold or whose area is less than a preset area threshold, and fill the filtered regions with background pixels to generate the extracted image.

[0131] The system filters out regions whose area ratio is less than a preset threshold or whose area is less than a preset area threshold. The threshold value is 1 / 30, and the area threshold value is 100. The filtering criteria are: regions with a threshold value less than 1 / 30 or an area less than 100 are selected, and the corresponding pixel blocks in the target foreground image are filled with background pixels to generate the extracted image.

[0132] However, the value of the ratio threshold in this embodiment is not limited to this. Depending on the specific application scenario, the ratio threshold can be (but is not limited to): 1 / 5, 1 / 10, 1 / 20, 1 / 27, or other user-defined custom values. Similarly, the value of the area threshold in this embodiment is not limited to this. Depending on the specific application scenario, the area threshold can be (but is not limited to): 10, 50, 70, 90, 130, 200, or other user-defined custom values.

[0133] In some implementations, the image enhancement strategy is to enhance the sharpness of the target foreground image. See also... Figure 5 , Figure 5 This is a schematic diagram of the process for enhancing the sharpness of the target foreground image in this embodiment.

[0134] like Figure 5 As shown, S1400 includes:

[0135] S1431. Read the pixels in the feature mask image sequentially;

[0136] Each pixel in the feature mask image is read, and each pixel is used as a read pixel in steps S1432-S1435. Therefore, the process of reading each pixel involves steps S1432-S1435, and then reading the next pixel, and so on, until every pixel in the feature mask image has been read, at which point the reading operation ends.

[0137] S1432. Generate an image selection area based on the read pixel as the center and a preset radius value;

[0138] A circle is drawn with the read pixel as the center and a set radius value. The resulting image is the image selection area. In this embodiment, the set radius value is 3. However, the radius value is not limited to this. Depending on the specific application scenario, it can be set to any value to adapt to the needs of the scenario.

[0139] S1433. Read the pixel values ​​in the image selection area and determine whether at least one pixel value in the image selection area is equal to a preset second pixel threshold.

[0140] Read the pixel value of each pixel in the image selection area and determine whether each pixel in the image selection area contains a pixel value of the second pixel threshold, where the second pixel threshold is 0. A pixel value of 0 represents no image in the image. When one or more pixel values ​​of 0 appear in the image selection area, it means that the read pixel is a boundary point of the target foreground image.

[0141] S1434. When at least one pixel value in the selected image area is equal to a preset second pixel threshold, the pixel value of the read pixel remains unchanged.

[0142] When at least one pixel value in the selected image area is equal to a preset second pixel threshold, it indicates that the read pixel is a boundary point of the target foreground image. In this case, the pixel value of the read pixel remains unchanged. When none of the pixel values ​​in the selected image area are 0, it indicates that the read pixel is not a boundary point of the target foreground image. In this case, the pixel value of the read pixel is filled with a fill pixel value. In this embodiment, the fill pixel value is 1. However, the fill pixel value is not limited to this. Depending on the specific scenario, the fill pixel value can be any value between 0 and 255.

[0143] S1435. Based on the read pixels, outline the target selection area in the target foreground image to generate the extracted image.

[0144] Based on the selected contour pixels, a contour map representing the target selection area in the target foreground image can be drawn according to the correspondence between the feature mask image and the target foreground image pixels. This makes the boundaries of the extracted image clearer.

[0145] In some implementations, the image enhancement strategy includes performing thin-band restoration processing on the target foreground image. See also... Figure 6 , Figure 6 This is a schematic diagram of the process for thin band restoration of the target foreground image in this embodiment.

[0146] like Figure 6 As shown, S1400 includes:

[0147] S1441. Input the target image into a preset thin band detection model, wherein the thin band detection model is a neural network model pre-trained to a convergent state for detecting images;

[0148] In this embodiment, "thin stripe" refers to the image lines of clothing straps or other thin lines when the target object image in the target image is clothing.

[0149] The thin band detection model detects the smallest rectangular region containing thin bands in the target image. The thin band detection module uses the Mask R-CNN model. However, the basic model of the thin band detection model is not limited to this. In some implementations, the thin band detection model can (but is not limited to): convolutional neural network model, deep convolutional neural network model, recurrent neural network model, or a variant of the above three models.

[0150] The initial model is trained to a convergent state through supervised or semi-supervised training. The model that has been trained to a convergent state is the thin band detection model.

[0151] The target image is input into a preset thin band detection model to generate a detection image of the target image.

[0152] S1442. Input the detection image output by the thin band detection model into a preset thin band image segmentation model, wherein the thin band image segmentation model is a neural network model pre-trained to a convergent state for thinning and segmenting images.

[0153] The detected image output by the thin band detection model is input into the thin band image segmentation model, which includes: u 2 The net model and the Sobel operator, the u 2 The net model comprises four U-block modules. Each U-block module includes a downsampling channel and an upsampling channel. The downsampling channel includes two 3x3 convolutional layers and multiple 2x2 convolutional layers. The input of each of the multiple 2x2 convolutional layers includes the outputs of the two adjacent convolutional layers above it. The thin band image is the u-block image. 2 A fused image segmented by the Net model and the Sobel operator. The thin-strip image segmentation model employs an improved u... 2 The net model architecture. Because deep feature maps offer almost no help in extracting thin bands, the original u... 2 The deepest two U-block (RSU) layers of the net model were removed, retaining only the first four U-block (RSU) layers. The number of downsampling and upsampling operations for each U-block layer was also reduced by one. This makes the network more lightweight, improving runtime efficiency without compromising segmentation performance.

[0154] Please see Figure 7 , Figure 7 This is a flowchart of the residual U-shaped unit (RSU) in this embodiment. Figure 7 As shown, based on the characteristics of thin bands, the segmented regions are generally quite long and narrow, for u 2The residual U-block (RSU) module in the .NET network has been improved. The kernel size of the convolutions in the deep feature maps (all feature maps after downsampling the input network image twice) has been changed from 3x3 to 2x2. Simultaneously, the number of input feature maps has been doubled, with the doubling derived from the input downsampling or linear transformation of the previous layer. This allows the model to focus more on thin bands and less on the background when extracting features, thus improving the model's feature extraction capabilities. Double-headed arrows indicate downsampling, and double-headed arrows indicate upsampling.

[0155] S1443. Read the thin strip image output by the thin strip image segmentation model, and perform image fusion processing on the thin strip image and the target foreground image to generate the extracted image.

[0156] This paper uses the Sobel operator to segment the detected thin band region and fuses the segmentation result with the output of the thin band image segmentation model as the final output of the thin band segmentation module. The fusion formula is: MSK = Wu * UMSK * SMSK + UMSK; the data in the above formula are normalized to between 0 and 1, where MSK is the final output of the thin band segmentation module, UMSK is the output of the thin band image segmentation model, SMSK is the output of the Sobel operator, and Wu is the weight of the Sobel operator output. This weight can be a static value or obtained by joint training with the thin band image segmentation model. In this paper, Wu is 1-UMSK. The reason for this setting is that, generally speaking, the accuracy of the thin band image segmentation model is relatively high. The segmentation result of the Sobel operator is only used as a supplement to the thin band image segmentation model to increase the recall of the thin band image segmentation model, that is, to repair missing or blurred thin bands. At the same time, it is not desirable to introduce impurities while increasing recall. This setting can effectively shield the impurities in the results of traditional segmentation, and only enhance the weak parts of the thin band image segmentation model.

[0157] It should be noted that in this embodiment, the noise removal processing, the sharpness enhancement processing, and the thin band restoration processing of the target foreground image can be performed sequentially, namely, the first step is the process of S1311-S1313, then the process of S1411-S1414 is performed, and finally the process of S1441-S1443 is performed to generate the extracted image.

[0158] Please refer to details. Figure 8 , Figure 8 This is a schematic diagram of the basic structure of the image processing device in this embodiment.

[0159] like Figure 8As shown, an image processing apparatus includes: an acquisition module 1100, a segmentation module 1200, a processing module 1300, and an enhancement module 1400. The acquisition module 1100 acquires a target image to be processed, wherein the target image includes a target selection region representing a preset target object image; the segmentation module 1200 inputs the target image into a preset segmentation model, wherein the segmentation model is a neural network model pre-trained to convergence for image extraction and segmentation; the processing module 1300 reads the feature mask image of the target selection region output by the segmentation model, and performs masking processing on the feature mask image and the target image to generate a target foreground image of the target image; the enhancement module 1400 performs image enhancement processing on the target foreground image according to a preset image enhancement strategy to generate an extracted image of the target image.

[0160] The image processing device generates a feature mask image of the target object in the target image by performing image segmentation. Using this feature mask image as a mask for the target image, image filtering and selection are performed to extract the target foreground image representing the target object. The target foreground image is then enhanced to generate an extracted image of the target selection region. Because the edge color difference values ​​of the feature mask image are more obvious, it is easier to distinguish the difference between the target selection region and the background pixels. After using the segmented and extracted feature mask image for mask filtering, the resulting target foreground image is more complete, leading to a more complete and clear final extracted image.

[0161] In some embodiments, the image processing apparatus further includes: a first mask submodule, a first operation submodule, and a first processing submodule. The first mask submodule is used to generate a mask matrix of the target image based on the feature mask image; the first operation submodule is used to multiply the mask matrix with the image matrix of the target image to generate a foreground matrix; and the first processing submodule is used to perform pixelation processing on the foreground matrix to generate the target foreground image.

[0162] In some embodiments, the image enhancement strategy includes: performing noise removal processing on the target foreground image. The image processing device further includes: a first filling submodule, a second processing submodule, a first connectivity submodule, and a first generation submodule. Specifically, the first filling submodule fills the background region of the target foreground image with preset background pixels to generate a first foreground image; the second processing submodule binarizes the first foreground image according to a preset first pixel threshold to generate a binary image of the first foreground image; the first connectivity submodule performs connected component processing on the binary image, connecting pixels with the same pixel value in the binary image to generate multiple region blocks; and the first generation submodule performs area filtering on the multiple region blocks to generate the extracted image.

[0163] In some embodiments, the image processing apparatus further includes: a first extraction submodule, a second calculation submodule, and a first filtering submodule. The first extraction submodule is used to extract the region block with the largest area from the plurality of region blocks as a reference region block; the second calculation submodule is used to perform proportional calculations between the areas of each of the plurality of region blocks and the reference region block to generate an area ratio of each region block to the reference region block; the first filtering submodule is used to filter region blocks whose area ratio is less than a preset ratio threshold or whose area is less than a preset area threshold, and fill the filtered region blocks with background pixels to generate the extracted image.

[0164] In some embodiments, the image enhancement strategy includes: enhancing the sharpness of the target foreground image. The image processing device further includes: a first reading submodule, a second generation submodule, a first confirmation submodule, a second filling submodule, and a third generation submodule. Specifically, the first reading submodule is used to sequentially read pixels in the feature mask image; the second generation submodule is used to generate an image selection area with the read pixels as centers and a preset radius value; the first confirmation submodule is used to read pixel values ​​in the image selection area and determine whether at least one pixel value in the image selection area is equal to a preset second pixel threshold; the second filling submodule is used to fill the read pixel values ​​with a preset filling pixel value if at least one pixel value in the image selection area is equal to the preset second pixel threshold, indicating that the pixel is near the boundary; otherwise, the third generation submodule is used to outline the target selection area in the target foreground image based on the read pixels, generating the extracted image.

[0165] In some embodiments, the image enhancement strategy includes performing thin-strip restoration processing on the target foreground image. The image processing device further includes a third processing submodule, a fourth processing submodule, and a fourth generation submodule. The third processing submodule is used to input the target image into a preset thin-strip detection model, wherein the thin-strip detection model is a neural network model pre-trained to convergence for detecting images. The fourth processing submodule is used to input the detected image output by the thin-strip detection model into a preset thin-strip image segmentation model, wherein the thin-strip image segmentation model is a neural network model pre-trained to convergence for thinning image segmentation. The fourth generation submodule is used to read the thin-strip image output by the thin-strip image segmentation model and perform image fusion processing between the thin-strip image and the target foreground image to generate the extracted image.

[0166] In some embodiments, the thin-band image segmentation model includes: u 2 The net model and the Sobel operator, the u 2 The net model comprises four U-block modules. Each U-block module includes a downsampling channel and an upsampling channel. The downsampling channel includes two 3x3 convolutional layers and multiple 2x2 convolutional layers. The input of each of the multiple 2x2 convolutional layers includes the outputs of the two adjacent convolutional layers above it. The thin band image is the u-block image. 2 A fused image of images segmented by the Net model and the Sobel operator.

[0167] To address the aforementioned technical problems, embodiments of the present invention also provide a computer device. Please refer to [link / reference needed]. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.

[0168] like Figure 9 The diagram shows the internal structure of a computer device. The computer device includes a processor, non-volatile storage medium, memory, and a network interface connected via a system bus. The non-volatile storage medium stores the operating system, database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, they enable the processor to implement an image processing method. The processor provides computational and control capabilities, supporting the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to perform an image processing method. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0169] In this embodiment, the processor is used to execute... Figure 8 The image processing device includes a memory for the specific functions of the acquisition module 1100, segmentation module 1200, processing module 1300, and enhancement module 1400. The memory stores the program code and various types of data required to execute these modules. A network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all sub-modules in the image processing device, and the server can call the server's program code and data to execute the functions of all sub-modules.

[0170] Computer equipment generates a feature mask image of the target object in the target image by performing image segmentation. Using this feature mask image as a mask for the target image, image filtering and selection are performed to extract the target foreground image representing the target object. This foreground image is then enhanced to generate an extracted image of the target selection region. Because the edge color difference values ​​of the feature mask image are more obvious, it is easier to distinguish the difference between the target selection region and the background pixels. After using the feature mask image obtained from segmentation and extraction to perform mask filtering on the target image, the resulting target foreground image is more complete, leading to a more complete and clear final extracted image.

[0171] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the image processing method of any of the above embodiments.

[0172] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0173] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the image processing method of any of the above embodiments.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0175] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those disclosed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0176] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized by, Applied to an e-commerce platform scene, comprising: Collecting a target image to be processed, wherein the target image includes a target selection area representing a preset target object image; Inputting the target image into a preset segmentation model, wherein the segmentation model is a neural network model pre-trained to a convergence state for extracting and segmenting images, and the images used by the corresponding training set of the segmentation model are binary feature mask images, and the images output by the segmentation model are grayscale feature mask images; Reading the feature mask image of the target selection area output by the segmentation model, and performing mask processing on the feature mask image and the target image to generate a target foreground image of the target image, including: generating a mask matrix of the target image according to the feature mask image; multiplying the mask matrix and the image matrix of the target image to generate a foreground matrix; and performing pixelization processing on the foreground matrix to generate the target foreground image; According to a preset image enhancement strategy, performing image enhancement processing on the target foreground image to generate an extracted image of the target image, wherein the image enhancement strategy includes performing thin band recovery processing on the target foreground image; The thin band recovery processing includes: inputting the target image into a preset thin band detection model, the thin band detection model is a neural network model pre-trained to a convergence state for detecting images; inputting the detection image output by the thin band detection model into a preset thin band image segmentation model, the thin band image segmentation model is a neural network model pre-trained to a convergence state for refining and segmenting images; reading the thin band image output by the thin band image segmentation model, and performing image fusion processing on the thin band image and the target foreground image to generate the extracted image; the thin band image segmentation model includes a u2net model and a sobel operator, the u2net model includes 4 U-block modules, and the down-sampling channel of each U-block module contains two 3*3 convolution layers and multiple 2*2 convolution layers; The thin band image segmentation model further includes: each U-block module includes the down-sampling channel and an up-sampling channel, the input of each convolution layer in the multiple 2*2 convolution layers in the down-sampling channel includes the outputs of the two adjacent convolution layers, and the thin band image is a fusion image of the u2net model and the sobel operator segmentation image.

2. The image processing method of claim 1, wherein, The image enhancement strategy further includes: performing noise removal processing on the target foreground image, specifically including: Filling a preset background pixel in a background area of the target foreground image to generate a first foreground image; Performing binaryzation processing on the first foreground image according to a preset first pixel threshold to generate a binary image of the first foreground image; Performing connected domain processing on the binary image to make the pixel points with the same pixel value in the binary image regionally connected to generate multiple region blocks; Performing area screening on the multiple region blocks to generate the extracted image.

3. The image processing method of claim 2, wherein, The area screening of the plurality of region blocks generates the extraction image, and includes: extracting a region block with the largest area in the plurality of region blocks as a reference region block; performing proportional operation on the area of each region block in the plurality of region blocks and the reference region block to generate an area ratio of each region block to the reference region block; screening region blocks with an area ratio less than a preset proportional threshold or an area less than a preset area threshold in the area ratios, and filling the screened region blocks as background pixels to generate the extraction image.

4. The image processing method of claim 1, wherein, The image enhancement strategy further includes: a target foreground image definition enhancement process, specifically including: reading pixel points in the feature mask image in sequence; generating an image selection area according to a preset radius value with the read pixel points as the center; reading pixel values in the image selection area and determining whether at least one of the pixel values in the image selection area is equal to a preset second pixel threshold value; when at least one of the pixel values in the image selection area is equal to the preset second pixel threshold value, the pixel point is a boundary pixel, the pixel value is unchanged, otherwise the pixel value of the read pixel point is filled as a preset filling pixel value; according to the read pixel point, the outline of the target selection area in the target foreground image is outlined to generate the extraction image.

5. An image processing apparatus characterized by comprising: Applied to an e-commerce platform scene, including: a collection module configured to collect a target image to be processed, wherein the target image includes a target selection area representing a preset target object image; a segmentation module configured to input the target image into a preset segmentation model, wherein the segmentation model is a neural network model pre-trained to a convergent state and configured to perform extraction and segmentation processing on images, and the images used in a corresponding training set of the segmentation model are binary feature mask images, and the images output by the segmentation model are grayscale feature mask images; a processing module configured to read a feature mask image of the target selection area output by the segmentation model, and perform mask processing on the feature mask image and the target image to generate a target foreground image of the target image, including: generating a mask matrix of the target image according to the feature mask image; multiplying the mask matrix and an image matrix of the target image to generate a foreground matrix; and performing pixelization processing on the foreground matrix to generate the target foreground image; The enhancement module is used for image enhancement processing of the target foreground image according to a preset image enhancement strategy, and generates an extraction image of the target image, wherein the image enhancement strategy comprises fine band recovery processing of the target foreground image; the fine band recovery processing comprises: inputting the target image into a preset fine band detection model, the fine band detection model being a neural network model pre-trained to a convergence state and used for image detection; inputting a detection image output by the fine band detection model into a preset fine band image segmentation model, the fine band image segmentation model being a neural network model pre-trained to a convergence state and used for fine segmentation of an image; reading a fine band image output by the fine band image segmentation model, and performing image fusion processing on the fine band image and the target foreground image to generate the extraction image; the fine band image segmentation model comprises a u2net model and a sobel operator, the u2net model comprises four U-block modules, and a down-sampling channel of each U-block module comprises two 3*3 convolution layers and a plurality of 2*2 convolution layers; the fine band image segmentation model further comprises: each U-block module comprises the down-sampling channel and an up-sampling channel, an input of each convolution layer in the plurality of 2*2 convolution layers in the down-sampling channel comprises outputs of the two adjacent convolution layers, and the fine band image is a fusion image of the u2net model and the sobel operator.

6. The image processing apparatus according to claim 4, characterized by Further comprising: The first filling submodule is used for filling a preset background pixel to a background region of the target foreground image to generate a first foreground image; The second processing submodule is used for performing binaryzation processing on the first foreground image according to a preset first pixel threshold to generate a binary image of the first foreground image; The first connection submodule is used for performing connected domain processing on the binary image to make pixel points with the same pixel value in the binary image regionally connected to generate a plurality of region blocks; The first generation submodule is used for performing area screening on the plurality of region blocks to generate the extraction image.

7. The image processing apparatus according to claim 5, characterized by, Further comprising: The first extraction submodule is used for extracting a region block with the largest area in the plurality of region blocks as a reference region block; The second operation submodule is used for performing proportional operation on areas of the region blocks in the plurality of region blocks and the reference region block respectively to generate area ratios of the region blocks to the reference region block; The first screening submodule is used for screening region blocks with an area ratio less than a preset proportion threshold or an area less than a preset area threshold in the area ratios, and filling the screened region blocks as background pixels to generate the extraction image.

8. The image processing apparatus according to claim 5, characterized by, Further comprising: The first reading submodule is used for sequentially reading pixel points in the feature mask image; The second generation submodule is used for generating an image selection area according to a preset radius value with the read pixel points as the center; The first confirmation submodule is used for reading pixel values in the image selection area, and determining whether at least one of the pixel values in the image selection area is equal to a preset second pixel threshold; a second filling submodule, configured to fill the pixel value of the read pixel point with a preset filling pixel value when the pixel value of at least one of the pixels in the image selected area is equal to a preset second pixel threshold value, the pixel point being a pixel near the boundary, and the pixel value being unchanged, otherwise; a third generation submodule, configured to outline the contour of the target selected area in the target foreground image according to the read pixel point, and generate the extracted image.

9. A computer device, comprising: An image processing device, comprising a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions are executed by the processor to make the processor execute the steps of the image processing method according to any one of claims 1 to 4.

10. A storage medium storing computer readable instructions, wherein, The computer readable instructions are executed by one or more processors to make the one or more processors execute the steps of the image processing method according to any one of claims 1 to 4.

Images