Image mask recognition, extraction, model training method, device and electronic equipment

Abstract

The embodiment of the present invention provides a method, device and electronic device for image mask recognition, extraction and model training, wherein the image mask recognition method includes: extracting features from an image to be recognized and a target image reference area selected by a user in the image to be recognized; determining the feature similarity between the first feature data corresponding to the feature diffusion source and the second feature data corresponding to the feature diffusion destination; superimposing the first feature data and the second feature data according to the weights of the feature similarity identifiers; and performing image mask recognition based on the feature data corresponding to the image to be recognized after the weighted superposition processing to generate an image mask recognition result. The embodiment of the present invention introduces a feature diffusion mechanism, uses the target image reference area selected by the user as a feature diffusion source, and performs feature diffusion in the image to be recognized, thereby improving the accuracy of target object recognition in the image to be recognized and the efficiency of image segmentation processing.

Description

Technical Field

[0001] The present application relates to a method, device and electronic equipment for identifying, extracting and training a model of an image mask, and belongs to the field of computer technology. Background Art

[0002] Separation masks are widely used in image processing. For example, they can be applied to some cutout tools to separate specific objects in a picture from the image background using image masking technology.

[0003] However, in the prior art, the segmentation mask annotation processing of images often relies on a large amount of manual stroking or smearing to complete the segmentation mask recognition of different objects in the image. In many cases, the user needs to mark the target image reference points a considerable number of times to obtain a relatively satisfactory segmentation mask, thereby reducing the efficiency of image segmentation and extraction. Summary of the invention

[0004] The embodiment of the present invention provides a method, device and electronic device for image mask recognition, extraction and model training to improve the recognition efficiency of image masks.

[0005] In order to achieve the above object, an embodiment of the present invention provides a method for recognizing an image mask, comprising:

[0006] Extracting features from the image to be identified and the target image reference area selected by the user in the image to be identified;

[0007] Taking the target image reference area as a feature diffusion source and one or more areas outside the target image reference area as feature diffusion destinations, determining a feature similarity between first feature data corresponding to the feature diffusion source and second feature data corresponding to the feature diffusion destinations;

[0008] superimposing the first feature data and the second feature data according to the weight of the feature similarity identifier;

[0009] Image mask recognition is performed according to the feature data corresponding to the image to be recognized after the weighted superposition processing to generate a recognition result of the image mask.

[0010] The embodiment of the present invention also provides a method for training an image segmentation model, comprising:

[0011] Using a plurality of training images as images to be recognized, obtaining a target image reference area selected by a user for a target object, and using the aforementioned image mask recognition method to obtain an image mask for the target object;

[0012] The training image and the corresponding image mask are used as training data to train a machine learning model, and the machine learning model is used to perform image segmentation processing to segment the target object from the specified image.

[0013] The embodiment of the present invention further provides an image mask recognition device, comprising:

[0014] A feature extraction module, used for extracting features from the image to be identified and the target image reference area selected by the user in the image to be identified;

[0015] a feature similarity determination module, configured to use the target image reference area as a feature diffusion source and one or more areas outside the target image reference area as feature diffusion destinations, and determine a feature similarity between first feature data corresponding to the feature diffusion source and second feature data corresponding to the feature diffusion destinations;

[0016] A feature diffusion module, used for superimposing the first feature data and the second feature data according to the weight of the feature similarity identifier;

[0017] The recognition processing module is used to perform image mask recognition according to the feature data corresponding to the image to be recognized after the weighted superposition processing, and generate a recognition result of the image mask.

[0018] The embodiment of the present invention further provides a training device for an image segmentation model, comprising:

[0019] An image mask acquisition module is used to use multiple training images as images to be recognized, obtain a target image reference area selected by a user for a target object, perform feature diffusion processing using the target image reference area as a feature diffusion source, perform image mask recognition on the image to be recognized after the feature diffusion processing, and obtain an image mask for the target object;

[0020] A training processing module is used to train a machine learning model using the training image and the corresponding image mask as training data, wherein the machine learning model is used to perform image segmentation processing to segment the target object from the specified image.

[0021] An embodiment of the present invention further provides an electronic device, including:

[0022] Memory, used to store programs;

[0023] The processor is used to run the program stored in the memory to execute the above-mentioned image mask recognition method.

[0024] An embodiment of the present invention further provides an electronic device, including:

[0025] Memory, used to store programs;

[0026] A processor is used to run the program stored in the memory to execute the aforementioned image segmentation model training method.

[0027] The embodiment of the present invention also provides an image extraction method, comprising:

[0028] Use the image segmentation model to extract the input image for the specified target image;

[0029] In response to a user's correction operation on the cut-out target image, obtaining a target image reference area selected by the user;

[0030] The target image reference area is used as a feature diffusion source to perform feature diffusion processing, image mask recognition is performed on the image to be identified after the feature diffusion processing, the image mask of the target object is obtained, and the input image is re-extracted for the specified target image using the image mask to generate a new target image.

[0031] An embodiment of the present invention further provides an electronic device, including:

[0032] Memory, used to store programs;

[0033] The processor is used to run the program stored in the memory to execute the above-mentioned image extraction method.

[0034] The image mask recognition, extraction, model training method, device and electronic device of the embodiment of the present invention introduce a feature diffusion mechanism, use the target image reference area selected by the user as the feature diffusion source, perform feature diffusion processing based on feature similarity in the image to be recognized, enhance the feature significance of image points similar to the feature diffusion source selected by the user, thereby improving the accuracy of the image recognition model in recognizing the target object in the image to be recognized, thereby reducing the number of times the user selects the target image reference point, and improving the efficiency of image segmentation processing.

[0035] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of an exemplary structure of an image mask recognition and processing system according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of the process of the image mask recognition method according to an embodiment of the present invention;

[0038] Figure 3 A flowchart of a training method for an image segmentation model according to an embodiment of the present invention;

[0039] Figure 4 Schematic diagram of the structure of an image mask recognition device according to an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of the structure of a training device for an image segmentation model according to an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of an application scenario of the image extraction method according to an embodiment of the present invention;

[0042] Figure 7 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0044] An embodiment of the present invention proposes a method for recognizing an image mask. By introducing feature diffusion and pixel diffusion mechanisms, foreground points and / or background points selected by a user are used as diffusion sources and diffused in the image to be recognized, thereby improving the accuracy of the image recognition model in recognizing the target object in the image to be recognized. As a result, the user only needs to select the foreground points and / or background points a small number of times to mark the image mask area corresponding to the target object, thereby improving the efficiency and accuracy of image segmentation processing.

[0045] like Figure 1 As shown, it is an exemplary structural diagram of the image mask recognition processing system of an embodiment of the present invention. The processing system can be deployed on a cloud server to provide users with cloud platform-based image processing services. Users can interact with the cloud server through image processing tools set on terminal devices, such as various cutout tools. Users can display the image to be recognized on the terminal device and select the target image reference area by clicking on the image to be recognized. In addition, the above-mentioned processing system can also be directly deployed on the terminal device.

[0046] The target image reference area mentioned here can be realized by the user's selection of the target image reference point. After the target image reference point is selected, the area of ​​the preset range near the reference point is used as the target image reference area. The target image reference point can include the foreground point and / or background point selected by the user as the determination reference of the target object image. In the image processing process, the foreground point and / or background point selected by the user is used as the subsequent determination reference for other areas in the image as foreground points or background points, that is, the determination reference of the target object. Among them, the foreground point is in the target object range that the user wants to extract, and the background point is in the area outside the target object range that the user wants to extract. The image mask is a template used to cover the specified target area in the image, and is generally used to separate the image in the specified target area from the image to be processed. In digital image processing, the image mask can be specifically a two-dimensional matrix array (such as a mask matrix) or an image template for filtering the specified area. The processing process based on the image mask is to use the selected image or graphic to block the processed image (all or part) to control the area or processing process of the image processing. In specific applications, image mask recognition is generally applied to segment a specified target in an image. For example, if you want to segment an image of a dog from a photo containing a background environment, you need to identify the image area corresponding to the dog target and form an image mask. When segmenting a specific target, the point in the image area where the target is located is the foreground point, and the point in the image area where the target is located is the background point. Users can trigger the image recognition model to determine the image area where the target is located by continuously selecting foreground points and / or background points, thereby forming an image mask and performing image segmentation.

[0047] Theoretically, the user can trigger the image recognition model to determine the accurate target image area by selecting foreground points and / or background points countless times, but this will seriously waste the user's time and be extremely inefficient. In the embodiment of the present invention, a diffusion mechanism based on the target image reference points selected by the user is introduced to improve the processing efficiency of mask recognition.

[0048] like Figure 1As shown in , after the user selects the foreground point and / or the background point, the Gaussian kernel processing module will be used to process and generate a Gaussian map representation of the foreground point and / or the background point. Specifically, a predetermined range of pixel areas containing the foreground point and / or the background point is used as the target image reference area, and a Gaussian kernel with a preset amplitude and standard deviation (for example, a Gaussian kernel with an amplitude of 1 and a standard deviation of 10) is used to process the target image reference area to generate a Gaussian map corresponding to the target image reference area, and then the Gaussian map and the image to be identified are input into the feature extraction module together to perform feature extraction. Among them, the main function of using the Gaussian kernel for processing is to perform noise reduction processing on the exchange point area, so that accurate feature information corresponding to the local area can be obtained. Specifically, two Gaussian maps can be generated for the foreground point image reference area and the background point image reference area corresponding to the foreground point and the background point, respectively, and spliced ​​with the image to be identified with three channels of RGB (three primary color light mode) to form a 5-channel image as the input of the feature extraction module. In addition, the feature extraction module can be implemented using the feature extraction part of the DeeplabV3+ (Deep Atrous Network Enhanced Version 3) segmentation model. DeeplabV3+ is a semantic segmentation model. It uses an encoder-decoder structure for reasoning. The encoder is used to extract abstract high-level semantic information, and the decoder is used to restore image detail information. In the encoder part, DeeplabV3+ uses atrous convolution to expand the receptive fields while maintaining the scale of the feature map, so that the model can obtain background information from a larger range for reasoning, thereby obtaining better high-level semantic representation. In the decoder part, DeeplabV3+ introduces the fusion of the underlying feature map and the high-level feature map to reconstruct the image detail features. As a result, the prediction results of DeeplabV3+ have both high-level semantic features and underlying detail features.

[0049] The feature data generated after being processed by the feature extraction module is input into the feature diffusion module for feature diffusion processing, so that the features of the reference area of ​​the target image are diffused to the area similar to it. Specifically, the feature diffusion processing may include the following processing steps:

[0050] 1) Constructing feature diffusion sources: using the foreground point image reference area and / or the background point image reference area as the foreground feature diffusion source and / or the background feature diffusion source, respectively, wherein the feature to be diffused is the feature vector corresponding to the target image reference area.

[0051] 2) Constructing feature diffusion destinations: Based on the foreground point image reference area and / or background point image reference area determined in 1), use the image recognition model to make preliminary predictions of the foreground points and background points for each pixel in the entire image, and determine whether each pixel is used as a foreground feature diffusion destination or a background feature diffusion destination based on the preliminary prediction results.

[0052] 3) Calculate feature similarity: Calculate the feature similarity between the feature vector of the feature diffusion source and the feature vector of the feature diffusion destination through vector dot product processing.

[0053] 4) Feature diffusion: weighted according to the feature similarity calculated in 3), that is, using the feature similarity as the weight, the feature vector corresponding to the feature diffusion source is weightedly superimposed on the feature vector corresponding to the feature diffusion destination. Given that the foreground feature diffusion destination and the background feature diffusion destination have been determined in 2), the feature vectors corresponding to the foreground feature diffusion source and the background feature diffusion source can be weightedly superimposed with the feature vectors corresponding to the foreground feature diffusion destination and the background feature diffusion destination, thereby achieving feature diffusion.

[0054] After feature diffusion processing, it is equivalent to updating and adjusting the feature data of the entire image to be identified. Based on the feature data of the image to be identified after feature diffusion processing, the image recognition model is used to predict the foreground and background points of each pixel in the whole image, and an intermediate prediction result is generated. The intermediate prediction result is used for subsequent pixel diffusion processing, wherein the intermediate prediction result mentioned here includes the prediction probability of each pixel being a foreground point and / or background point. The image recognition model here can still use the above-mentioned DeeplabV3+ segmentation model.

[0055] After the above intermediate prediction results are input to the pixel diffusion module, the pixel diffusion module performs diffusion processing from the foreground point and / or background point selected by the user to the surrounding pixel points based on the similarity of the pixel data, so as to adjust the prediction probability of the foreground point and / or background point corresponding to the diffused pixel point. Specifically, the pixel diffusion processing may include the following processing process:

[0056] 1) Constructing pixel diffusion sources: Selecting pixels in the foreground point image reference area and / or the background point image reference area as foreground pixel diffusion sources and / or background pixel diffusion sources. The pixel can be one or more pixels, or a pixel area composed of multiple pixels can be used as a pixel diffusion source. The content of pixel diffusion is the pixel data corresponding to the pixel, such as some color, brightness, saturation and other data based on RGB data.

[0057] 2) Constructing pixel diffusion destinations: The pixel diffusion destinations may be selected from the pixel points around the pixel diffusion source, that is, the pixels are diffused from the pixel diffusion source to the surrounding pixel points in a radiating form.

[0058] 3) Calculate pixel similarity: Calculate the data difference of the RGB channels between the pixel diffusion source and the pixel diffusion destination as the pixel similarity. The smaller the difference, the higher the pixel similarity.

[0059] 4) Pixel diffusion: Based on the pixel similarity in 3), the prediction probability of the foreground point and / or background point of the pixel diffusion source in the intermediate prediction result is weighted and superimposed on the prediction probability of the foreground point and / or background point of the pixel diffusion destination, thereby adjusting the prediction result of the pixel point. The weighted processing mentioned here is based on the size of the pixel similarity. The greater the pixel similarity, the greater the influence of the pixel diffusion source on the prediction result of the pixel diffusion destination during the superposition process.

[0060] 5) Iteration processing: In practical applications, it is necessary to repeatedly perform the pixel diffusion processing from 1) to 4) above, that is, after a round of pixel diffusion processing, the pixel point corresponding to the pixel diffusion destination of this round is used as a new pixel diffusion source, and the pixel diffusion is continued to the surrounding area according to the above processing to expand the influence range of the initial pixel diffusion source. The number of iterations can be preset as needed, and the higher the number of iterations, the wider the pixel diffusion range.

[0061] Through pixel diffusion processing, it is possible to adjust the prediction results of pixels within a certain range, that is, to fine-tune the intermediate prediction results after feature diffusion. This fine-tuning effect will be more obvious at the mask boundary.

[0062] It should be noted that the above-mentioned feature diffusion processing and pixel diffusion processing are essentially weighted accumulation processing based on similarity, and their purpose is to propagate the feature information of the diffusion source to the diffusion destination, so as to make more full use of the influence of the target image reference point selected by the user, improve the accuracy of target mask recognition, and thus reduce the number of times the user needs to select the target image reference point.

[0063] After the intermediate prediction results are adjusted by the pixel diffusion module, they are input into the threshold determination module. Based on the preset confidence threshold, the prediction probability of foreground points and / or background points in the intermediate prediction results is determined, thereby outputting the final prediction result of whether each pixel point is a foreground point or a background point, thereby determining the mask area of ​​the target object. Among them, the confidence threshold can be set according to actual needs, and the confidence determination module can be implemented using the Sigmoid function. The Sigmoid function is a threshold function with a value range of (0,1). It can map a real number to the interval of (0,1) and can be used for binary classification.

[0064] It should be noted that the above-mentioned intermediate prediction results can also be output as final recognition results. Even if only feature diffusion processing is used without pixel diffusion processing, the technical solution of the embodiment of the present invention can achieve the technical effect of reducing the number of times users select foreground points and / or background points and improving the efficiency and accuracy of image segmentation processing compared to the prior art.

[0065] The image mask recognition method proposed in the embodiment of the present invention introduces a feature diffusion mechanism, uses the area where the foreground point and / or background point selected by the user is located as a feature diffusion source, and performs feature diffusion in the image to be recognized, thereby improving the accuracy of the image recognition model in recognizing the target object in the image to be recognized, and then performs pixel diffusion based on the pixel data of the pixel points in the area where the selected foreground point and / or background point is located, thereby fine-tuning the prediction result, and further improving the accuracy of the prediction probability of the pixel points in the image to be recognized. The solution of the embodiment of the present invention allows the user to mark the image mask area corresponding to the target object only by selecting a small number of foreground points and / or background points, thereby improving the efficiency and accuracy of image segmentation processing.

[0066] The image mask recognition method of the embodiment of the present invention can be applied to generate training data for a machine learning model for image segmentation processing. In some application scenarios, users need a tool for cutting out a certain type of target object, and in order to realize the automatic recognition of such a cutting out tool, it is generally completed with the help of machine learning. For example, a clothing design company may need to cut out clothing images in a large number of pictures on the Internet to obtain various clothing data to assist in design and industry analysis. In response to this demand, a machine learning model for identifying target objects in pictures can be provided to users, such as a machine learning model based on a deep neural network. Users can train the machine learning model according to their own needs, so that the machine learning model can meet the user's purpose. The basis for users to reasonably train the machine learning model is to be able to construct a large amount of training data. The image mask recognition method provided by the embodiment of the present invention can be used for the processing of generating training data. Users can collect a large number of pictures as training images, and then use the method provided by the embodiment of the present invention to obtain a more accurate image mask of the target object by selecting the foreground point and / or background point of the target object once or multiple times. The training image and the corresponding image mask are then used as training data to train the machine learning model, and the trained machine learning model is used to perform image segmentation processing to segment the target object from the specified image. For example, if a user uses a large amount of training data that uses clothes as the target object, the machine learning model can automatically identify the clothes in the specified image and segment them out. The advantage of using the method provided by the embodiment of the present invention is that a large amount of training data can be generated quickly and accurately, thereby reducing the cost of collecting training data.

[0067] In terms of product application, the image mask recognition method provided by the embodiment of the present invention can perform data annotation according to user needs. By the user performing a small number of click operations on the image area where the target object is located and / or the image area of ​​the non-target object, a high-quality image mask can be obtained, thereby efficiently completing the data annotation process and generating training data. In this way, a large amount of training data can be generated very conveniently and quickly, so as to train the image segmentation model in the cutout tool to meet the user's cutout needs. In addition, using the method provided by the embodiment of the present invention, the image segmentation processing result can also be corrected, and then, based on the corrected image mask, the image segmentation model is trained to improve the accuracy of the image segmentation model for the target object. On the whole, the image mask recognition method, image extraction method, and model training method provided by the embodiment of the present invention can realize the entire process system from "annotated data-training model-model prediction-result correction", reduce the collection cost of model training data, and facilitate the user to correct the model prediction result, that is, to correct the cutout result, and the corrected result can also be fed back for training, thereby further improving the accuracy of the image segmentation model, which can better meet the user's cutout needs.

[0068] In addition, in the process of executing the image mask recognition method or image extraction processing of the embodiment of the present invention, it is possible to prompt selectable click positions or areas based on the preliminary analysis of the image. For example, based on the recognition of each target object in the image, an indicator mark that prompts the user to select can be generated at the boundary of the object image. In addition, the image mask recognition method or image extraction processing method of the embodiment of the present invention can be embedded in the processing of image editing and production, that is, as part of the image editing and production tools. For example, in the production process of some promotional posters, it may be necessary to remove some objects in the original image or extract some objects from some photographic works for later synthesis processing, etc. These applications can use the method provided by the embodiment of the present invention.

[0069] The technical solution of the present invention is further illustrated by some specific embodiments below.

[0070] Embodiment 1

[0071] like Figure 2 As shown, it is a flow chart of the image mask recognition method of an embodiment of the present invention. The method can be applied to a cloud service platform, and the user interacts with the cloud service platform through a terminal device to realize interactive recognition processing for the image mask. In addition, the method can also be run on a terminal device to complete the interactive recognition processing of the image mask locally. Specifically, the method includes:

[0072] S101: Feature extraction is performed on the image to be identified and the target image reference area selected by the user in the image to be identified. The target image reference point is a point selected by the user as a feature reference, which may include a foreground point and / or a background point. In the image processing process, the foreground point and / or the background point selected by the user is used as a reference for determining whether other areas in the image are foreground points or background points. The foreground point is within the target range that the user wants to extract, while the background point is outside the target range that the user wants to extract. In actual processing, an area within a specified range containing the target image reference point is generally selected as a feature reference. This area is the so-called target image reference area, and its size can be set as needed.

[0073] The feature extraction of the image to be identified and the target image reference area can be implemented by using a neural network model for image recognition processing, for example, it can be implemented by using the feature extraction part in the DeeplabV3+ segmentation model. For the processing of the target image reference area, a Gaussian kernel function can be used for noise reduction and other processing to obtain more accurate feature information as a subsequent feature diffusion source. Specifically, the feature extraction of the target image reference area selected by the user in the image to be identified can include: obtaining the foreground point and / or background point selected by the user; using the predetermined range of pixel areas containing the foreground point and / or background point as the target image reference area and performing feature extraction. The feature extraction process can be specifically: using a Gaussian kernel function for processing, generating a Gaussian map of the target image reference area, and then, performing feature extraction on the Gaussian map. In practical applications, two Gaussian maps can be generated for the foreground point image reference area and the background point image reference area corresponding to the foreground point and background point selected by the user, respectively, and spliced ​​with the image to be identified with three RGB channels to form a 5-channel image as input data for feature extraction processing.

[0074] S102: Taking the target image reference area as a feature diffusion source and one or more areas outside the target image reference area as feature diffusion destinations, determine the feature similarity between the first feature data corresponding to the feature diffusion source and the second feature data corresponding to the feature diffusion destination. Specifically, the first feature data and the second feature data can be specifically a feature vector corresponding to the feature diffusion source and a feature vector corresponding to the feature diffusion destination. Specifically, the feature similarity between the feature vector of the feature diffusion source and the feature vector of the feature diffusion destination can be calculated by processing a vector dot product.

[0075] In addition, as described above, the user can select foreground points and / or background points, and accordingly, a target image reference area including a foreground point image reference area and / or a background point image reference area will be formed. Accordingly, the process of determining the feature diffusion destination may include: predicting the foreground points and / or background points according to the feature data of the foreground point image reference area / or the background point image reference area and the feature data of the image to be identified, and generating a preliminary prediction result; determining the feature diffusion destination corresponding to the foreground point image reference area and / or the background point image reference area according to the preliminary prediction result. The preliminary prediction result mentioned here is the prediction of the foreground points and / or background points based on the image to be identified before the feature diffusion. Through this preliminary prediction, the probability of each point in the image to be identified being a foreground point and / or a background point can be roughly judged, so that targeted diffusion can be performed, that is, for the part with the predicted probability tending to the foreground point, for example, the probability of being a foreground point is greater than 0.5, it is used as the feature diffusion destination corresponding to the foreground point image reference area. Similarly, for the part with the predicted probability tending to the background point, it is used as the feature diffusion destination corresponding to the foreground point image reference area. Through such processing, the feature diffusion processing can be made more effective.

[0076] S103: Superimpose the first feature data and the second feature data according to the weights of the feature similarity identifiers. Specifically, the feature vector corresponding to the feature diffusion source can be weighted and superimposed on the feature vector corresponding to the feature diffusion destination according to the weights of the feature similarity identifiers. By weighted superposition of feature vectors based on feature similarity, the features at the foreground points and / or background points selected by the user can be diffused to the entire image to be identified, thereby making the user's selection of the target image reference point each time more effective.

[0077] S104: Perform image mask recognition based on the feature data of the image to be recognized after weighted superposition processing to generate a recognition result of the image mask. The image mask recognition can be implemented using the above-mentioned neural network model for image recognition processing, for example, using the DeeplabV3+ segmentation model to perform image mask recognition.

[0078] After the feature diffusion process, it is equivalent to updating and adjusting the feature data of the entire image to be identified. Based on the feature data of the image to be identified after the feature diffusion process, the image recognition model is used to predict the foreground points and background points of each pixel of the whole image, and the prediction results of each pixel are generated. The prediction results mentioned here include the prediction probabilities of each pixel being a foreground point and / or a background point. Then, the prediction probabilities of each pixel being a foreground point and / or a background point are compared with the preset confidence threshold, and the recognition results of the pixels in the image to be identified can be determined, thereby determining the mask area of ​​the target object. This process can be implemented by the aforementioned Sigmoid function.

[0079] Through the above-mentioned feature diffusion processing, the similarity information between the features of the target image reference area selected by the user and the features of other areas is fully utilized, the feature data of the entire image to be identified is adjusted, and then the image mask recognition is performed. In this way, compared with the recognition results of the prior art, its accuracy has been substantially improved, so that the user can use fewer target image reference point selection times to complete the image mask recognition.

[0080] In order to further improve the accuracy of mask recognition, the prediction result obtained based on the feature data of the image to be recognized after feature diffusion can be further subjected to pixel diffusion processing. That is, step S104 can further include:

[0081] S1041: Predict the foreground point and / or background point according to the feature data of the image to be identified after the weighted superposition processing, and generate an intermediate prediction result. That is, the prediction result of the foreground point and / or background point mentioned above is not directly output, but is used as an intermediate prediction result for subsequent pixel diffusion processing.

[0082] S1042: Taking the pixel points in the target image reference area as the pixel diffusion source, taking the pixel points around the pixel diffusion source as the pixel diffusion destination, and performing weighted superposition processing on the intermediate prediction results with the weights of the pixel similarity identifiers between the pixel diffusion source and the pixel diffusion destination. This step may further include the following iterative process:

[0083] S10421: Determine the pixel similarity between the pixel diffusion source and the pixel diffusion destination based on the pixel data of the pixel point. Specifically, the data difference of the RGB channels between the pixel diffusion source and the pixel diffusion destination may be calculated as the pixel similarity.

[0084] S10422: Using the weights identified by the pixel similarity, the prediction results of the pixel diffusion source are weighted and superimposed on the prediction results of the pixel diffusion destination.

[0085] S10423: Use the current pixel diffusion destination as the new pixel diffusion source, and the pixel points around the new pixel diffusion source as the new pixel diffusion destination, and repeat the pixel similarity calculation and weighted superposition processing of the prediction results until the preset round is reached, that is, reset S10421 and S10423 until the preset round. The more rounds are executed, the larger the range of pixel diffusion and the better the pixel diffusion effect. In actual applications, a balance can be struck between processing efficiency and accuracy improvement, and a reasonable iteration round can be selected.

[0086] S1043: Generate the recognition result of the image mask according to the intermediate prediction result after weighted superposition processing. After the pixel diffusion processing is performed, it is equivalent to adjusting the prediction probability of many pixel points. Finally, based on the prediction probability of the pixel point being a foreground point and / or a background point in the intermediate prediction result after weighted superposition processing, it is compared with the preset confidence threshold to determine the recognition result of the pixel point in the image to be recognized. Similarly, this part of the processing can be implemented by the aforementioned Sigmoid function.

[0087] The above describes the image mask recognition process performed after the user selects a target image reference point. The target image reference point selection mentioned here may include the user selecting multiple foreground points and / or background points. The user can trigger the image mask recognition process of the embodiment of the present invention by pressing the confirmation button. The image mask recognition process is essentially based on the user's selection of the foreground point and / or background point of the judgment reference to predict the foreground point and / or background point of each pixel point in the whole image, and then determine the range of the image mask according to the prediction result.

[0088] In practical applications, multiple selections may be required, that is, the accurate range of the image mask can be determined by executing the method provided by the embodiment of the present invention multiple times. Therefore, the above method may also include: in response to the user's operation of selecting the target image reference area in the image to be identified again, using the re-selected target image reference area as the feature diffusion source and one or more areas outside the target image reference area as the feature diffusion destination, calculating the feature similarity between the feature vector corresponding to the feature diffusion source and the feature vector corresponding to the feature diffusion destination, and performing subsequent processing.

[0089] The image mask recognition method proposed in the embodiment of the present invention introduces a feature diffusion mechanism, uses the area where the foreground point and / or background point selected by the user is located as a feature diffusion source, and performs feature diffusion in the image to be recognized, thereby improving the accuracy of the image recognition model in recognizing the target object in the image to be recognized. Then, based on the pixel data corresponding to the foreground point and / or background point selected by the user, pixel diffusion is performed to fine-tune the prediction result, further improving the accuracy of the prediction probability of the pixel point in the image to be recognized. The solution of the embodiment of the present invention allows the user to mark the image mask area corresponding to the target object only by selecting a small number of foreground points and / or background points, thereby improving the efficiency and accuracy of image segmentation processing.

[0090] Embodiment 2

[0091] like Figure 3As shown, it is a flow chart of the training method of the image segmentation model of an embodiment of the present invention. The method can be applied to a cloud service platform. The user interacts with the cloud service platform through a terminal device to realize the collection of training data and the training processing of the image segmentation model. In addition, the method can also be run on a terminal device to complete the above processing locally. Specifically, the method includes:

[0092] S201: Use multiple training images as images to be identified, obtain the target image reference area selected by the user for the target object, use the target image reference area as a feature diffusion source to perform feature diffusion processing, perform image mask recognition on the image to be identified after the feature diffusion processing, and obtain the image mask for the target object. For the specific mask recognition processing based on feature diffusion, the image mask recognition method provided in the aforementioned embodiment can be used to obtain the image mask for the target object. The target object mentioned here can represent a class of objects, for example, clothes can be used as the target object, or animals can be used as the target object, or a specific action such as a dog can be used as the target object. The selection of the target object also determines the specific application of the machine learning model obtained after training.

[0093] S202: Using the training image and the corresponding image mask as training data, training a machine learning model, the machine learning model is used to perform image segmentation processing to segment the target object from the specified image. The machine learning model can be, for example, a machine learning model based on a deep neural network, and the user can train the machine learning model according to their own needs, so that the machine learning model can meet the user's purpose. The trained machine learning model can be embedded in the user's cutout tool or web tool, so as to provide the user with personalized automatic image segmentation processing.

[0094] Furthermore, since the trained machine learning model may also have inaccurate recognition, in this case, the recognition result can still be corrected using the image mask recognition method provided by the embodiment of the present invention. Specifically, the method of this embodiment may further include:

[0095] S203: Use the machine learning model to perform image segmentation processing to generate a result image after segmentation processing. The result image can be displayed to the user through the terminal device. If the user finds that the target object segmentation is inaccurate, he can perform subsequent correction operations.

[0096] S204: In response to the user's correction operation on the result image, the target image reference area selected by the user is obtained, the result image is used as the image to be identified, and the result image is corrected using the image mask recognition method provided in the embodiment of the present invention. The user makes corrections based on the result image after the segmentation process is performed, and the correction method is to select the foreground point and / or background point as the target image reference point. This situation is the same as the situation mentioned in the previous embodiment, where the user needs to perform multiple target image reference point selection operations to obtain an accurate image mask. The corrected result image can be displayed to the user again through the terminal device. If there are still inaccuracies, the user can perform the correction process again.

[0097] The corrected result image can be used as training data to train the machine learning model in the form of online feedback, thereby updating the parameters of the machine learning model and further improving the accuracy. Specifically, the following step S205 can be performed.

[0098] S205: Obtain an image mask after correcting the result image, and use the corrected image mask and the corresponding original image as training data to train the machine learning model. The original image here refers to the image input into the machine learning model for image segmentation processing.

[0099] It can be seen that through the training method of the image segmentation model of the embodiment of the present invention, a large amount of accurate training data can be efficiently generated. In addition, the processing results can be corrected during the use of the model, and the model can be updated and trained, thereby continuously improving the accuracy of image segmentation processing.

[0100] Embodiment 3

[0101] like Figure 4 As shown, it is a schematic diagram of the structure of the image mask recognition device of an embodiment of the present invention. The device can be applied to a cloud service platform, and a user interacts with the cloud service platform through a terminal device to realize interactive recognition processing for an image mask. In addition, the device can also be run on a terminal device to complete the interactive recognition processing of an image mask locally. Specifically, the device includes:

[0102] The feature extraction module 11 is used to extract features from the image to be identified and the target image reference area selected by the user in the image to be identified. The target image reference point is a point selected by the user as a feature reference, which may include a foreground point and / or a background point. The foreground point is within the target range that the user wants to extract, while the background point is outside the target range that the user wants to extract. In actual processing, an area within a specified range containing the target image reference point is generally selected as a feature reference. This area is the so-called target image reference area, and its size can be set as needed.

[0103] The feature extraction of the image to be identified and the target image reference area can be implemented by using a neural network model for image recognition processing, for example, it can be implemented by using the feature extraction part of the DeeplabV3+ segmentation model. For the processing of the target image reference area, a Gaussian kernel function can be used for noise reduction and other processing to obtain more accurate feature information as a subsequent feature diffusion source. Specifically, feature extraction of the target image reference area selected by the user in the image to be identified can include: obtaining the foreground point and / or background point selected by the user; using a predetermined range of pixel areas containing the foreground point and / or background point as the target image reference area and performing feature extraction. The feature extraction process can be specifically: using a Gaussian kernel function for processing, generating a Gaussian map of the target image reference area, and then performing feature extraction on the Gaussian map.

[0104] The feature similarity determination module 12 is used to determine the feature similarity between the first feature data corresponding to the feature diffusion source and the second feature data corresponding to the feature diffusion destination, using the target image reference area as the feature diffusion source and one or more areas outside the target image reference area as the feature diffusion destination. Specifically, the first feature data and the second feature data can be specifically a feature vector corresponding to the feature diffusion source and a feature vector corresponding to the feature diffusion destination. Specifically, the feature similarity between the feature vector of the feature diffusion source and the feature vector of the feature diffusion destination can be calculated by processing the vector dot product.

[0105] In addition, as described above, the user can select a foreground point and / or a background point, and accordingly, an image reference area including a foreground point image reference area and / or a background point image reference area will be formed. Accordingly, the process of determining the feature diffusion destination may include: predicting the foreground point and / or the background point according to the feature data of the foreground point image reference area and / or the background point image reference area and the feature data of the image to be identified, and generating a preliminary prediction result; and determining the feature diffusion destination corresponding to the foreground point image reference area and / or the background point image reference area according to the preliminary prediction result.

[0106] The feature diffusion module 13 is used to superimpose the first feature data and the second feature data according to the weights of the feature similarity identifiers. Specifically, the feature vector corresponding to the feature diffusion source can be weighted and superimposed on the feature vector corresponding to the feature diffusion destination according to the weights of the feature similarity identifiers. By weighted superposition of feature vectors based on feature similarity, the features at the foreground points and / or background points selected by the user can be diffused to the entire image to be identified, so that the user's selection of the target image reference point each time is more effective.

[0107] The recognition processing module 14 is used to perform image mask recognition based on the feature data of the image to be recognized after the weighted superposition processing, and generate a recognition result of the image mask. The image mask recognition can be implemented by using the above-mentioned neural network model for image recognition processing, for example, using the DeeplabV3+ segmentation model to perform image mask recognition.

[0108] After the feature diffusion process, it is equivalent to updating and adjusting the feature data of the entire image to be identified. Based on the feature data of the image to be identified after the feature diffusion process, the image recognition model is used to predict the foreground points and background points of each pixel of the whole image, and the prediction results of each pixel are generated. The prediction results mentioned here include the prediction probabilities of each pixel being a foreground point and / or a background point. Then, the prediction probabilities of each pixel being a foreground point and / or a background point are compared with the preset confidence threshold, and the recognition results of the pixels in the image to be identified can be determined, thereby determining the mask area of ​​the target object. This process can be implemented by the aforementioned Sigmoid function.

[0109] In order to further improve the accuracy of mask recognition, the prediction result obtained based on the feature data of the image to be recognized after feature diffusion can be further subjected to pixel diffusion processing. That is, the processing of the recognition processing module 14 can further include:

[0110] The pixel points in the target image reference area are used as pixel diffusion sources, and the pixel points around the pixel diffusion source are used as pixel diffusion destinations. The intermediate prediction results are weighted superposition processed with the weights of the pixel similarity identifiers between the pixel diffusion source and the pixel diffusion destination. The recognition result of the image mask is generated according to the intermediate prediction results after the weighted superposition process.

[0111] The above-mentioned weighted superposition processing of the intermediate prediction results with the weights identified by the pixel similarity between the pixel diffusion source and the pixel diffusion destination may include the following iterative process: taking the pixel points in the reference area of ​​the target image as the pixel diffusion source, taking the pixel points around the pixel diffusion source as the pixel diffusion destination, and calculating the pixel similarity between the pixel diffusion source and the pixel diffusion destination based on the pixel data of the pixel points. Specifically, the data difference of the RGB channel between the pixel diffusion source and the pixel diffusion destination can be calculated as the pixel similarity; the prediction result as the pixel diffusion source is weightedly superimposed on the prediction result of the pixel diffusion destination with the weight identified by the pixel similarity. Then, taking the current pixel diffusion destination as the new pixel diffusion source, taking the pixel points around the new pixel diffusion source as the new pixel diffusion destination, repeatedly performing the pixel similarity calculation and the weighted superposition processing of the prediction results until reaching the preset round, and after the iteration is completed, performing the above-mentioned processing of generating the recognition result of the image mask according to the intermediate prediction result after the weighted superposition processing.

[0112] After the pixel diffusion processing is performed, it is equivalent to adjusting the prediction probability of many pixel points. Finally, the prediction probability of the pixel point being a foreground point and / or a background point in the intermediate prediction result after the weighted superposition processing is compared with the preset confidence threshold to determine the recognition result of the pixel point in the image to be recognized.

[0113] The image mask recognition device proposed in the embodiment of the present invention introduces a feature diffusion mechanism, uses the area where the foreground point and / or background point selected by the user is located as a feature diffusion source, and performs feature diffusion in the image to be recognized, thereby improving the accuracy of the image recognition model in recognizing the target object in the image to be recognized. Then, based on the pixel data corresponding to the foreground point and / or background point selected by the user, pixel diffusion is performed to fine-tune the prediction result, further improving the accuracy of the prediction probability of the pixel point in the image to be recognized. The solution of the embodiment of the present invention allows the user to mark the image mask area corresponding to the target object only by selecting a small number of foreground points and / or background points, thereby improving the efficiency and accuracy of image segmentation processing.

[0114] Embodiment 4

[0115] like Figure 5 As shown, it is a structural diagram of the training device of the image segmentation model of an embodiment of the present invention. The device can be applied to a cloud service platform, and the user interacts with the cloud service platform through a terminal device to realize the collection of training data and the training processing of the image segmentation model. In addition, the device can also be run on a terminal device to complete the above processing locally. Specifically, the device includes:

[0116] The image mask acquisition module 21 is used to use multiple training images as images to be identified, obtain the target image reference area selected by the user for the target object, use the target image reference area as a feature diffusion source to perform feature diffusion processing, perform image mask recognition on the image to be identified after the feature diffusion processing, and obtain the image mask for the target object. For the specific mask recognition processing based on feature diffusion, the image mask recognition device of the aforementioned embodiment can be used to obtain the image mask for the target object. The target object mentioned here can represent a class of objects, for example, clothes can be used as the target object, or animals can be used as the target object, or a specific action such as a dog can be used as the target object. The selection of the target object also determines the specific application of the machine learning model obtained after training.

[0117] The training processing module 22 is used to train the machine learning model using the training image and the corresponding image mask as training data. The machine learning model is used to perform image segmentation processing to segment the target object from the specified image. The machine learning model can be, for example, a machine learning model based on a deep neural network. The user can train the machine learning model according to their own needs so that the machine learning model can meet the user's purpose. The trained machine learning model can be embedded in the user's cutout tool or web page tool to provide the user with personalized automatic image segmentation processing.

[0118] Furthermore, since the trained machine learning model may also have inaccurate recognition, in this case, the recognition result can still be corrected using the image mask recognition device provided by the embodiment of the present invention, and the corrected result image can be used as training data to train the machine learning model in the form of online feedback, thereby updating the parameters of the machine learning model and further improving the accuracy. The detailed processing process has been described in the aforementioned embodiment.

[0119] It can be seen that through the training method of the image segmentation model of the embodiment of the present invention, a large amount of accurate training data can be efficiently generated. In addition, the processing results can be corrected during the use of the model, and the model can be updated and trained, thereby continuously improving the accuracy of image segmentation processing.

[0120] Embodiment 6

[0121] like Figure 6 As shown, it is a schematic diagram of an application scenario of the image extraction method of an embodiment of the present invention. The device can be applied to a cloud service platform, and the user interacts with the cloud service platform through a terminal device to achieve image extraction processing. In addition, the device can also be run on a terminal device and embedded in a cutout tool to help the user perform cutout processing on a target object. Specifically, the method includes:

[0122] S301: Use the image segmentation model to perform cropping processing on the input image for the specified target image. In some application scenarios, the cloud service platform can provide users with a cropping tool based on a machine learning model, in which an image segmentation model trained according to user needs is embedded to help users crop the target image specified by the user. For example, as a user's clothing design company, it may be necessary to crop clothing images from a large number of pictures on the Internet. Therefore, the cloud service platform can provide users with an image segmentation model specifically used for cropping clothing images for the user's cropping processing.

[0123] Combination Figure 6 As shown, the cutout tool shown in the figure includes a trained image segmentation model and the above-mentioned image mask recognition device for generating an image mask through interaction with the user. The upper left box represents the input image for cutout processing, and the input image will be cutout by the image segmentation model, and the lower left image before correction will be output. As shown in the figure, the four-pointed star pattern in the image represents the target image to be cut out, but the target image may have some problems and needs further correction by the user.

[0124] S302: In response to the user's correction operation on the cut-out target image, the target image reference area selected by the user is obtained. In some cases, the image segmentation model may have deviations for the cut-out details. After seeing the cut-out result, the user can select some foreground points and / or background points through the aforementioned click interaction operation to mark the target image reference area for image correction.

[0125] Combination Figure 6 As shown, based on the interactive interface provided by the cutout tool, the user selects foreground points (solid points) and background points (hollow points) near the edge of the target image to be corrected (the dotted part shown in the figure) on the image before correction obtained in the previous step, thereby triggering the image mask recognition device provided in an embodiment of the present invention to generate an image mask.

[0126] S303: Use the image mask recognition method in the above-mentioned embodiment to obtain the image mask of the target object, and use the image mask to re-extract the input image for the specified target image to generate a new target image. Based on the target image reference area marked by the user in the previous step, the image mask can be regenerated, and then the image is cut out based on the new image mask to form a corrected new target image in the lower right corner. It can be seen from the figure that the edge of the new target image has changed. If the user is still not satisfied with the new target image, the above-mentioned step S302 can be repeated to continue to correct the target image until the target image that the user is satisfied with is extracted.

[0127] Combination Figure 6 As shown, the image mask generated by the image mask recognition device, the image segmentation model can re-perform the cutout operation and generate a corrected new cutout image. If the user is still not satisfied with the new cutout image, the user can return to the interactive interface provided by the cutout tool, reselect the foreground point and / or background point, trigger the generation of a new image mask, and re-perform the cutout process until the user is satisfied.

[0128] In addition, after generating a new target image that satisfies the user, the image segmentation model can be trained based on the target image that has been corrected and confirmed by the user and the corresponding original image as training data, thereby improving the accuracy of the image segmentation model in the cutout processing of the specified target image, thereby better meeting the needs of the user.

[0129] Embodiment 7

[0130] The above embodiments describe the process and device structure of image mask recognition processing, image extraction method and image segmentation model training processing. The functions of the above methods and devices can be realized by means of an electronic device, such as Figure 7 As shown, it is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which specifically includes: a memory 110 and a processor 120.

[0131] The memory 110 is used to store programs.

[0132] In addition to the above-mentioned programs, the memory 110 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0133] The memory 110 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0134] The processor 120 is coupled to the memory 110 and is used to execute the program in the memory 110 to perform the operating steps of the image mask recognition processing method and / or the image extraction method and / or the image segmentation model training processing method described in the above embodiments.

[0135] In addition, the processor 120 may also include the various modules described in the aforementioned embodiments to perform image mask recognition processing and / or image extraction methods and / or image segmentation model training processing, and the memory 110 may be used, for example, to store data required for these modules to perform operations and / or output data.

[0136] The specific description of the above-mentioned processing process, the detailed description of the technical principle and the detailed analysis of the technical effect have been described in detail in the previous embodiments and will not be repeated here.

[0137] Furthermore, as shown in the figure, the electronic device may also include other components such as a communication component 130, a power component 140, an audio component 150, and a display 160. The figure only schematically shows some components, which does not mean that the electronic device only includes the components shown in the figure.

[0138] The communication component 130 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component 130 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 130 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0139] The power supply component 140 provides power to various components of the electronic device. The power supply component 140 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0140] The audio component 150 is configured to output and / or input audio signals. For example, the audio component 150 includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 110 or sent via the communication component 130. In some embodiments, the audio component 150 also includes a speaker for outputting audio signals.

[0141] The display 160 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0142] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing an image mask, comprising: Extracting features from the image to be identified and the target image reference area selected by the user in the image to be identified; Taking the target image reference area as a feature diffusion source and one or more areas outside the target image reference area as feature diffusion destinations, determining a feature similarity between first feature data corresponding to the feature diffusion source and second feature data corresponding to the feature diffusion destinations; superimposing the first feature data and the second feature data according to the weight represented by the feature similarity; Performing image mask recognition according to the feature data corresponding to the image to be recognized after weighted superposition processing to generate a recognition result of the image mask; Among them, according to the feature data of the image to be identified after the weighted superposition processing, image mask recognition is performed to generate the recognition result of the image mask, which includes: predicting the foreground points and / or background points according to the feature data of the image to be identified after the weighted superposition processing to generate an intermediate prediction result; taking the pixel points in the reference area of ​​the target image as the pixel diffusion source, taking the pixel points around the pixel diffusion source as the pixel diffusion destination, and performing weighted superposition processing on the intermediate prediction result with the weight of the pixel similarity identifier between the pixel diffusion source and the pixel diffusion destination; and generating the recognition result of the image mask according to the intermediate prediction result after the weighted superposition processing.

2. The method according to claim 1, wherein: The weighted superposition processing of the intermediate prediction results includes: Determining pixel similarity between the pixel diffusion source and the pixel diffusion destination based on pixel data of the pixel point; The intermediate prediction result as the pixel diffusion source is weighted and superimposed on the intermediate prediction result of the pixel diffusion destination according to the weight of the pixel similarity identifier; The current pixel diffusion destination is used as a new pixel diffusion source, and the pixel points around the new pixel diffusion source are used as new pixel diffusion destinations. The pixel similarity calculation and the weighted superposition processing of the intermediate prediction results are repeatedly performed until a preset round is reached.

3. The method according to claim 1, wherein: The feature extraction of the target image reference area selected by the user in the image to be recognized includes: Obtaining foreground points and / or background points selected by the user; Taking a predetermined range of pixel areas including the foreground points and / or background points as the target image reference area; Feature extraction is performed on the target image reference area.

4. The method according to claim 3, wherein: Extracting features from the target image reference area includes: A Gaussian kernel function is used for processing to generate a Gaussian map of the reference area of ​​the target image, and feature extraction is performed on the Gaussian map.

5. The method according to claim 2, wherein: Determining the pixel similarity between the pixel diffusion source and the pixel diffusion destination includes: The data difference value of the RGB channels between the pixel diffusion source and the pixel diffusion destination is determined as the pixel similarity.

6. The method according to claim 1, wherein: According to the intermediate prediction results after weighted superposition processing, the recognition results of the generated image mask include: The prediction probability of a pixel point being a foreground point and / or a background point in the intermediate prediction result after the weighted superposition processing is compared with a preset confidence threshold to determine the recognition result of the pixel point in the image to be recognized.

7. The method according to claim 1, wherein: The selected target image reference area includes a foreground point image reference area and / or a background point image reference area, and the method further includes: Predicting the foreground points and / or the background points based on the feature data of the foreground point image reference area / or the background point image reference area and the feature data of the image to be identified, and generating a preliminary prediction result; According to the preliminary prediction result, a feature diffusion destination corresponding to the foreground point image reference area and / or the background point image reference area is determined.

8. The method according to claim 1, wherein: Also includes: In response to the user's operation of reselecting a target image reference area in the image to be identified, the reselected target image reference area is used as a feature diffusion source, and one or more areas outside the target image reference area are used as feature diffusion destinations, the feature similarity between the first feature data corresponding to the feature diffusion source and the second feature data corresponding to the feature diffusion destination is determined, and subsequent processing is performed.

9. A method for training an image segmentation model, comprising: Using a plurality of training images as images to be recognized, obtaining a target image reference area selected by a user for a target object, and using the image mask recognition method according to any one of claims 1 to 8 to obtain an image mask for the target object; The training image and the corresponding image mask are used as training data to train a machine learning model, and the machine learning model is used to perform image segmentation processing to segment the target object from the specified image.

10. The method according to claim 9, wherein: Also includes: Perform image segmentation processing using the machine learning model to generate a result image after segmentation processing; In response to the user's correction operation on the result image, the target image reference area selected by the user is obtained, the result image is used as the image to be identified, and the result image is corrected using the image mask recognition method described in any one of claims 1 to 9.

11. The method according to claim 10, wherein: Also includes: Obtaining an image mask after correcting the result image; The modified image mask and the corresponding original image are used as training data to train the machine learning model.

12. An image mask recognition device, comprising: A feature extraction module, used for extracting features from the image to be identified and the target image reference area selected by the user in the image to be identified; a feature similarity determination module, configured to use the target image reference area as a feature diffusion source and one or more areas outside the target image reference area as feature diffusion destinations, and determine a feature similarity between first feature data corresponding to the feature diffusion source and second feature data corresponding to the feature diffusion destinations; A feature diffusion module, used for superimposing the first feature data and the second feature data according to the weight represented by the feature similarity; A recognition processing module, used to perform image mask recognition according to the feature data corresponding to the image to be recognized after weighted superposition processing, and generate a recognition result of the image mask; The method for performing image mask recognition based on the feature data of the image to be recognized after weighted superposition processing and generating the recognition result of the image mask comprises: predicting foreground points and / or background points based on the feature data of the image to be recognized after weighted superposition processing and generating an intermediate prediction result; taking the pixel points in the reference area of ​​the target image as pixel diffusion sources, taking the pixel points around the pixel diffusion source as pixel diffusion destinations, and performing weighted superposition processing on the intermediate prediction results with weights identified by the pixel similarity between the pixel diffusion source and the pixel diffusion destination; and generating the recognition result of the image mask based on the intermediate prediction result after the weighted superposition processing.

13. The device according to claim 12, wherein: According to the intermediate prediction results after weighted superposition processing, the recognition results of the generated image mask include: The prediction probability of a pixel point being a foreground point and / or a background point in the intermediate prediction result after the weighted superposition processing is compared with a preset confidence threshold to determine the recognition result of the pixel point in the image to be recognized.

14. A training device for an image segmentation model, comprising: An image mask acquisition module is used to use multiple training images as images to be recognized, obtain a target image reference area selected by a user for a target object, perform feature diffusion processing using the target image reference area as a feature diffusion source, perform image mask recognition on the image to be recognized after the feature diffusion processing, and obtain an image mask for the target object; A training processing module is used to train a machine learning model using the training image and the corresponding image mask as training data, wherein the machine learning model is used to perform image segmentation processing to segment the target object from the specified image.

15. An electronic device, comprising: Memory, used to store programs; A processor, configured to run the program stored in the memory to execute the image mask recognition method according to any one of claims 1 to 8.

16. An electronic device, comprising: Memory, used to store programs; A processor, configured to run the program stored in the memory to execute the image segmentation model training method according to any one of claims 9 to 11.

17. An image extraction method, comprising: Use the image segmentation model to extract the input image for the specified target image; In response to a user's correction operation on the cut-out target image, obtaining a target image reference area selected by the user; The image mask of the target image is obtained by using the image mask recognition method according to any one of claims 1 to 8, and the input image is re-processed for extracting the designated target image by using the image mask to generate a new target image.

18. The method according to claim 17, wherein: Also includes: The new target image and the corresponding original image are used as training data to train the image segmentation model.

19. An electronic device comprising: Memory, used to store programs; A processor, used to run the program stored in the memory to execute the image extraction method described in claim 17 or 18.

Images