General matting method and system based on artificial intelligence

Abstract

The invention provides a general matting method and system based on artificial intelligence, and relates to the technical field of picture processing, and the method comprises the following steps: S1, inputting an image; s2, image processing; s3, subject recognition and segmentation; s4, four-channel cutout output is carried out, and cutout results of the four channels are generated after segmentation is completed and are respectively red, green and blue (RGB) three-color channels and a transparency channel (Alpha channel); s5, outputting a result; the system comprises an input module, a preprocessing module, a subject identification and segmentation module, a four-channel matting output module, a result output module and a data set. The user only needs to provide one image containing the main body, the system can automatically complete the whole processing flow from input to output, no extra user input or manual operation is needed, and the matting process is greatly simplified.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a technology for automatically cutting out a subject in an input image based on a deep learning model. Background Art

[0002] Image matting is an important task in image processing, which refers to separating the main part of interest from the image to make it stand out from the background. Traditional matting methods mainly rely on manual annotation by users or semi-automatic processing with the help of specific algorithms. These methods usually have the following defects: Time-consuming and labor-intensive: Traditional methods require users to perform a lot of manual operations, especially for complex images. The cutout process is time-consuming and labor-intensive; Poor versatility: Many algorithms can only perform cutouts on specific categories of images (such as people), and are difficult to adapt to diverse image content; Low precision: When dealing with complex backgrounds or subjects with rich details, it is difficult to achieve accurate cutouts using traditional methods, especially in the edge areas of the subject, which are prone to burrs, loss or poor fusion.

[0003] However, with the development of deep learning technology, image segmentation methods based on convolutional neural networks (CNNs) have made significant progress in many fields. However, existing deep learning-based image segmentation technologies still face some challenges: High demand for model training: Training an efficient cutout model usually requires a large amount of accurately labeled data, which increases the difficulty of model development; Dependence on user input: Some image cutout methods require users to provide preliminary segmentation masks or other auxiliary inputs, which limits their automation and generality; Limited applicability: Some methods work well in certain scenarios but do not work well in different categories or complex scenarios. Therefore, we propose a universal cutout method that does not require any user-assisted input and can automatically cutout various subjects with high accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a general image cutting method and system based on artificial intelligence to solve the above-mentioned deficiencies in the technology.

[0005] In order to achieve the above object, the present invention provides the following technical solutions: A general image cutting method based on artificial intelligence comprises the following steps: S1, input image, the user provides an image containing a subject, and the user does not need to perform any preprocessing or auxiliary input on the image; S2, image processing, preprocessing the input image, including: 1) Image normalization: normalize the image pixel values ​​to a uniform range to facilitate subsequent neural network processing 2) Resizing: resizing the image to make it fit the input requirements of the neural network while retaining the key features of the image; 3) Color correction: convert and adjust the color space of the image to enhance the image's expressiveness and recognizability; S3, subject recognition and segmentation, uses a deep neural network model to automatically recognize and segment the subject in the preprocessed image, introduces an attention mechanism to enhance the model's ability to separate the subject from the background, and combines feature maps of different scales to enable the model to handle subjects of different sizes and shapes; S4, four-channel matting output, after the segmentation is completed, a four-channel matting result is generated, which are red, green, blue (RGB) three-color channels and a transparency channel (Alpha channel); S5. Result output: The final four-channel image is output to the user. The output supports multiple formats, such as PNG and TIFF. These formats can retain the information of the Alpha channel, so that the output image can be directly used in various application scenarios.

[0006] Preferably, the image content includes but is not limited to various objects such as commodities, people, animals, plants and articles.

[0007] Preferably, the three-color channel generation: extracting RGB information of the subject according to the segmentation result and separating it from the background; The transparency channel generation: generates a corresponding Alpha channel for indicating the separation between the subject and the background, and through refined processing, ensures a smooth transition of the Alpha channel at the edge of the subject to avoid burrs or poor fusion.

[0008] The present invention also provides a general image cutting system based on artificial intelligence, comprising an input module, a preprocessing module, a subject recognition and segmentation module, a four-channel image cutting output module, a result output module and a data set; The input module is used to process various types of image content and is responsible for receiving image files uploaded by users. After the image is imported, it is immediately passed to the preprocessing module. The preprocessing module performs preliminary processing on the input image to optimize the performance of the subsequent deep learning model. The subject recognition and segmentation module uses a pre-trained deep neural network to perform subject segmentation on the image, including object detection, feature extraction and semantic segmentation, and finally generates a binary or multi-channel segmentation mask, which indicates the precise position of the subject in the image and uses a convolutional neural network layer to extract multi-level features of the image. The four-channel cutout output module uses the mask map generated by the segmentation module to cut out the subject from the background and generate a four-channel image containing an alpha channel. The alpha channel provides transparency information for each pixel, so that the cut-out image can achieve seamless background replacement and synthesis in subsequent applications. The alpha channel is generated according to the result of subject segmentation. The transition of the alpha channel is smoother by feathering the edges. Semi-transparent objects and gradient areas are processed, and the alpha blending technology is used to ensure that these areas present a real transparent effect in the final image. The result output module exports the generated four-channel image in a user-specified format and provides it to the user for download. It also provides image preview and batch processing functions and supports different output resolutions. The data set contains a large amount of image data from different fields to provide support for the general cut-out system.

[0009] Preferably, the input module receives the file through a file upload interface, and performs format check and preliminary verification in the background to ensure that the image meets the input requirements of the preprocessing module.

[0010] Preferably, the convolutional neural network layer extracts multi-level features of the image, including semantic information of edges, textures, colors, and object shapes, and improves the accuracy of segmentation of objects of different sizes through pyramid pooling, and handles complex edges such as hair and transparent objects.

[0011] Preferably, the subject recognition and segmentation module includes an edge perception module and a transparency perception module. In the processing of complex edges and transparent objects such as hair, the edge perception module and the transparency perception module are used to improve the segmentation accuracy.

[0012] Preferably, the input module, preprocessing module, subject recognition and segmentation module, four-channel cutout output module and result output module interact with each other through a data bus or an API interface, so that the modular design of the system can support the expansion and replacement of different functions.

[0013] Preferably, the data set performs the following processing on the data: Data labeling: To ensure the accuracy of the model, all training data has been carefully labeled to ensure that the subject area in each image is accurately marked; Model architecture: The CNN network structure is used, combined with ResNet as the encoder to ensure that the model has good feature extraction capabilities. At the same time, the attention mechanism and multi-scale feature fusion module are introduced to further improve the model's adaptability to different types of images; Training process: During the model training process, we used the transfer learning method, took the model pre-trained on large-scale image data as the initial model, and fine-tuned it on our dataset. During the training process, we used data enhancement technology to increase the diversity of the data and prevent the model from overfitting.

[0014] Preferably, after the data training is completed, the data set is optimized as follows: Accuracy optimization: By adjusting the weight of the loss function, the model can maintain high segmentation accuracy when processing images with complex edges; Computational efficiency optimization: During the model deployment phase, image compression, quantization, pruning and other technologies are used to reduce the amount of model calculations and improve operational efficiency; Fault-tolerance mechanism: In actual applications, if the system identifies multiple potential subjects in an image, it will generate multiple segmentation candidates, sort them according to the segmentation quality, and finally select the best result output.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. Strong versatility: The method of the present invention is applicable to various types of subject cutouts. Whether it is a commodity, a person, an animal, a plant or a car, the system can automatically identify and accurately segment it, and has wide applicability.

[0016] 2. High precision: By combining deep neural networks, attention mechanisms and multi-scale feature fusion technology, the present invention significantly improves the accuracy of subject segmentation, especially when processing images with complex backgrounds or rich details, and can effectively avoid burrs and poor edge fusion.

[0017] 3. Easy operation: The user only needs to provide an image containing the subject, and the system can automatically complete the entire processing flow from input to output without any additional user input or manual operation, which greatly simplifies the cutout process.

[0018] 4. High efficiency: Due to the use of efficient deep learning algorithms and optimized calculation processes, the present invention has a high processing speed while ensuring high precision, can quickly generate cutout results, and is suitable for large-scale image processing needs.

[0019] 5. Four-channel output: By outputting a four-channel image including an Alpha channel, the present invention can not only provide high-quality RGB information, but also retain fine transparency information, making the cutout result more realistic and natural, and convenient for secondary editing and use in various application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a general image cutting method based on artificial intelligence of the present invention; Figure 2 The system block diagram of the general image cutting system based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] To facilitate understanding of the present invention, the present invention will be described more comprehensively below with reference to related references, and several embodiments of the present invention are given. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this article includes any and all combinations of one or more related listed items.

[0024] Embodiment 1 like Figure 1 As shown, the present invention provides a technical solution: a general image cutting method based on artificial intelligence, comprising the following steps: S1, input image, the user provides an image containing a subject, and the user does not need to perform any preprocessing or auxiliary input on the image; S2, image processing, preprocessing the input image, including: 1) Image normalization: normalize the image pixel values ​​to a uniform range to facilitate subsequent neural network processing 2) Resizing: resizing the image to make it fit the input requirements of the neural network while retaining the key features of the image; 3) Color correction: convert and adjust the color space of the image to enhance the image's expressiveness and recognizability; S3, subject recognition and segmentation, uses a deep neural network model to automatically recognize and segment the subject in the preprocessed image, introduces an attention mechanism to enhance the model's ability to separate the subject from the background, and combines feature maps of different scales to enable the model to handle subjects of different sizes and shapes; S4, four-channel matting output, after the segmentation is completed, a four-channel matting result is generated, which are red, green, blue (RGB) three-color channels and a transparency channel (Alpha channel); S5. Result output: The final four-channel image is output to the user. It supports export in multiple formats, such as PNG and TIFF. These formats can retain the information of the Alpha channel, so that the output image can be directly used in various application scenarios, such as product display on e-commerce platforms, image publishing on social media, digital advertising design, etc.

[0025] The image content includes, but is not limited to, various objects such as commodities, people, animals, plants, and items.

[0026] Three-color channel generation: extract the RGB information of the subject based on the segmentation results and separate it from the background; Transparency channel generation: Generate the corresponding Alpha channel to indicate the separation between the subject and the background. Through fine processing, ensure the smooth transition of the Alpha channel at the edge of the subject to avoid burrs or poor fusion.

[0027] Embodiment 2 like Figure 2 As shown, based on the first embodiment, the present invention provides a general image cutting system based on artificial intelligence, including an input module, a preprocessing module, a subject recognition and segmentation module, a four-channel image cutting output module, a result output module and a data set; It should be noted that in order to realize the general cutout function of the present invention, a large-scale multi-category dataset is constructed, covering a large amount of image data from different fields, including but not limited to e-commerce product images, portraits, natural scenery, animal pictures and images of various complex scenes.

[0028] The input module is used to process various types of image content. It is responsible for receiving image files uploaded by users and supports a variety of common formats, such as JPEG, PNG, TIFF, etc. After the image is imported, it is immediately passed to the preprocessing module. The preprocessing module performs preliminary processing on the input image to optimize the performance of the subsequent deep learning model. The subject recognition and segmentation module uses a pre-trained deep neural network (CNN) to segment the image, including object detection, feature extraction and semantic segmentation, and finally generates a binary or multi-channel segmentation mask. The mask indicates the precise location of the subject in the image and uses the convolutional neural network layer to extract the multi-level features of the image. The four-channel cutout output module uses the mask map generated by the segmentation module to cut out the subject from the background and generate a four-channel image (RGBA) containing an Alpha channel. The Alpha channel provides transparency information for each pixel, so that the cutout image can achieve seamless background replacement and synthesis in subsequent applications. The Alpha channel is generated according to the result of the subject segmentation. The transition of the Alpha channel is smoother by feathering the edges. The semi-transparent objects and gradient areas are processed by Alpha blending technology (Alpha Blending) ensures that these areas present a real transparent effect in the final image. The result output module exports the generated four-channel image in a user-specified format (such as PNG, TIFF, etc.) and provides it to users for download. It also provides image preview and batch processing functions and supports different output resolutions. The dataset contains a large amount of image data from different fields to provide support for the general cutout system.

[0029] Among them, the preprocessing operations include image normalization, size adjustment, and color correction; Image normalization is to adjust the pixel value to a certain range, such as between 0 and 1; Resizing scales the image to the fixed input size required by deep learning models, such as 512x512 or 1024x1024; Color correction adjusts the color balance of an image to reduce noise or unusual color appearance.

[0030] The input module receives files through the file upload interface and performs format checks and preliminary verification in the background to ensure that the image meets the input requirements of the preprocessing module.

[0031] The convolutional neural network layer extracts multi-level features of the image, including semantic information of edges, textures, colors, and object shapes, and improves the accuracy of segmentation of objects of different sizes through pyramid pooling, and handles complex edges and transparent objects such as hair.

[0032] The subject recognition and segmentation module includes an edge perception module and a transparency perception module. When processing complex edges and transparent objects such as hair, the edge perception module and the transparency perception module are used to improve the segmentation accuracy.

[0033] The input module, preprocessing module, subject recognition and segmentation module, four-channel cutout output module and result output module interact with each other through a data bus or an API interface, so that the modular design of the system can support the expansion and replacement of different functions.

[0034] It should be added that the data set performs the following processing on the data: Data labeling: To ensure the accuracy of the model, all training data has been carefully labeled to ensure that the subject area in each image is accurately marked; Model architecture: The CNN network structure is used, combined with ResNet as the encoder to ensure that the model has good feature extraction capabilities. At the same time, the attention mechanism and multi-scale feature fusion module are introduced to further improve the model's adaptability to different types of images; Training process: During the model training process, we used the transfer learning method, took the model pre-trained on large-scale image data as the initial model, and fine-tuned it on our dataset. During the training process, we used data enhancement techniques such as flipping, rotation, and scaling to increase the diversity of the data and prevent the model from overfitting.

[0035] Furthermore, after the data training is completed, the data set is optimized as follows: Accuracy optimization: By adjusting the weights of loss functions, such as cross entropy loss and Dice loss, the model can maintain high segmentation accuracy when processing images with complex edges. Computational efficiency optimization: During the model deployment phase, image compression, quantization, pruning and other technologies are used to reduce the amount of model calculations and improve operational efficiency; Fault-tolerance mechanism: In actual applications, if the system identifies multiple potential subjects in an image, it will generate multiple segmentation candidates, sort them according to the segmentation quality, and finally select the best result output.

[0036] System flow of the method of the present invention: ① Input module: responsible for receiving image files uploaded by users and passing them to the preprocessing module; ② Preprocessing module: performs preprocessing operations such as image normalization, resizing, and color correction; ③ Subject recognition and segmentation module: core computing unit, using pre-trained deep neural network model for subject segmentation; ④ Four-channel cutout output module: generates a four-channel image including an Alpha channel based on the segmentation results; ⑤Result output module: exports the generated four-channel image into a specified format and provides it to users for download.

[0037] Based on the above process, the operating steps of the present invention are: the user uploads an image file containing a subject, the system automatically performs image preprocessing, and passes the processed image to the subject recognition and segmentation module, the system generates the segmentation result and generates a four-channel cutout image, the user selects the format of the output image and downloads the final cutout result.

[0038] In summary, in terms of application scenarios and example operations, the image cutout system of the present invention can be widely used in multiple fields. The following are several typical application scenarios: E-commerce platform: After users upload product pictures, the system can automatically generate product images without background, which can be used for product display, promotion and advertising design, saving the cost of manual image cutting; Social media: Users can quickly create personal avatars, social network covers, or other content that requires high-quality cutouts to enhance the visual effects of social media; Advertising design: Designers can use this system to quickly separate the subject, carry out creative design and background replacement, and improve work efficiency.

[0039] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A general image cutting method based on artificial intelligence, characterized in that: The following steps are involved: S1, input image, the user provides an image containing a subject, and the user does not need to perform any preprocessing or auxiliary input on the image; S2, image processing, preprocessing the input image, including: 1) Image normalization: normalize the image pixel values ​​to a uniform range to facilitate subsequent neural network processing 2) Resizing: resizing the image to make it fit the input requirements of the neural network while retaining the key features of the image; 3) Color correction: convert and adjust the color space of the image to enhance the image's expressiveness and recognizability; S3, subject recognition and segmentation, using a deep neural network model to automatically recognize and segment the subject in the preprocessed image, introducing an attention mechanism to enhance the model's ability to separate the subject from the background, and combining feature maps of different scales so that the model can handle subjects of different sizes and shapes; S4, four-channel matting output, after the segmentation is completed, a four-channel matting result is generated, which are red, green, blue (RGB) three-color channels and a transparency channel (Alpha channel); S5. Result output: The final four-channel image is output to the user. The output supports multiple formats, such as PNG and TIFF. These formats can retain the information of the Alpha channel, so that the output image can be directly used in various application scenarios.

2. The general image cutting method based on artificial intelligence according to claim 1, characterized in that: The image content includes but is not limited to various objects such as commodities, people, animals, plants and articles.

3. The general image cutting method based on artificial intelligence according to claim 1, characterized in that: The three-color channel generation: extracting the RGB information of the subject according to the segmentation result and separating it from the background; The transparency channel generation: generates a corresponding Alpha channel for indicating the separation between the subject and the background, and through refined processing, ensures a smooth transition of the Alpha channel at the edge of the subject to avoid burrs or poor fusion.

4. A general image cutting system based on artificial intelligence, used to implement a general image cutting method based on artificial intelligence as described in claims 1-3, characterized in that: It includes input module, preprocessing module, subject recognition and segmentation module, four-channel cutout output module, result output module and data set; The input module is used to process various types of image content and is responsible for receiving image files uploaded by users. After the image is imported, it is immediately passed to the preprocessing module, which performs preliminary processing on the input image to optimize the performance of the subsequent deep learning model. The subject recognition and segmentation module uses a pre-trained deep neural network (CNN) to perform subject segmentation on the image, including object detection, feature extraction and semantic segmentation, and finally generates a binary or multi-channel segmentation mask, which indicates the precise location of the subject in the image and uses a convolutional neural network layer to extract the multi-level features of the image. The four-channel cutout output module uses the mask map generated by the segmentation module to cut out the subject from the background and generate a four-channel image containing an alpha channel. The Alpha channel provides transparency information for each pixel, so that the cut-out image can achieve seamless background replacement and synthesis in subsequent applications. The Alpha channel is generated according to the result of subject segmentation. The transition of the Alpha channel is smoother by feathering the edges. Semi-transparent objects and gradient areas are processed, and Alpha blending technology is used to ensure that these areas present a real transparent effect in the final image. The result output module exports the generated four-channel image in a format specified by the user and provides it to the user for download. It also provides image preview and batch processing functions and supports different output resolutions. The data set contains image data from different fields to provide support for the general cut-out system.

5. A general image cutting system based on artificial intelligence according to claim 4, characterized in that: The input module receives files through the file upload interface and performs format check and preliminary verification in the background to ensure that the image meets the input requirements of the preprocessing module.

6. A general image cutting system based on artificial intelligence according to claim 4, characterized in that: The convolutional neural network layer extracts multi-level features of the image, including semantic information of edges, textures, colors, and object shapes, and improves the accuracy of segmentation of objects of different sizes through pyramid pooling.

7. A general image cutting system based on artificial intelligence according to claim 4, characterized in that: The subject recognition and segmentation module includes an edge perception module and a transparency perception module. In the processing of complex edges and transparent objects such as hair, the edge perception module and the transparency perception module are used to improve the segmentation accuracy.

8. A general image cutting system based on artificial intelligence according to claim 4, characterized in that: The input module, preprocessing module, subject recognition and segmentation module, four-channel cutout output module and result output module interact with each other through a data bus or an API interface, so that the modular design of the system can support the expansion and replacement of different functions.

9. A general image cutting system based on artificial intelligence according to claim 4, characterized in that: The data set performs the following processing on the data: Data labeling: To ensure the accuracy of the model, all training data has been carefully labeled to ensure that the subject area in each image is accurately marked; Model architecture: The CNN network structure is used, combined with ResNet as the encoder to ensure that the model has good feature extraction capabilities. At the same time, the attention mechanism and multi-scale feature fusion module are introduced to further improve the model's adaptability to different types of images; Training process: During the model training process, we used the transfer learning method, took the model pre-trained on large-scale image data as the initial model, and fine-tuned it on our dataset. During the training process, we used data enhancement technology to increase the diversity of the data and prevent the model from overfitting.

10. A general image cutting system based on artificial intelligence according to claim 9, characterized in that: After the data training is completed, the data set is optimized as follows: Accuracy optimization: By adjusting the weight of the loss function, the model can maintain high segmentation accuracy when processing images with complex edges; Computational efficiency optimization: During the model deployment phase, image compression, quantization, pruning and other technologies are used to reduce the amount of model calculations and improve operational efficiency; Fault-tolerance mechanism: In actual applications, if the system identifies multiple potential subjects in an image, it will generate multiple segmentation candidates, sort them according to the segmentation quality, and finally select the best result output.

Images