Image segmentation method and device, electronic equipment and storage medium
By inputting the target image to two different image segmentation models and generating the final segmentation result based on the overlapping parts of the results, the problem of low image segmentation accuracy is solved, and higher image segmentation accuracy is achieved.
Patent Information
- Application Number
- CN202311813537.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-26
- Publication Date
- 2025-06-27
AI Technical Summary
When performing image segmentation based on image segmentation model, there is a problem of low accuracy in image segmentation.
By inputting the target image to two different image segmentation models, the first segmentation result and the second segmentation result are obtained respectively, and the final target segmentation result is obtained based on the overlapping results between the two results.
This improves the accuracy of image segmentation, avoids the problem of segmentation inaccuracy caused by a single image segmentation model, and takes into account the segmentation performance of the two models.
Smart Images
Figure CN120219733A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of image segmentation, and in particular, to an image segmentation method, apparatus, electronic device, and storage medium. Background Art
[0002] At present, with the continuous development of artificial intelligence technology, image segmentation models have been widely used in fields such as intelligent image matting, face recognition, pedestrian detection, traffic control, and medical imaging, and have the advantages of high efficiency and high degree of automation. For example, an image can be input into an image segmentation model, and the image segmentation model outputs a segmentation result. However, in related technologies, when performing image segmentation based on an image segmentation model, there is a problem of low accuracy of image segmentation. Summary of the Invention
[0003] The present disclosure provides an image segmentation method, apparatus, electronic device, computer-readable storage medium, and computer program product to at least solve the problem of low accuracy of image segmentation when performing image segmentation based on an image segmentation model in related technologies. The technical solution of the present disclosure is as follows:
[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image segmentation method, including: obtaining a target image; inputting the target image into a first image segmentation model, and outputting a first segmentation result of the target image by the first image segmentation model; inputting the target image into a second image segmentation model, and outputting a second segmentation result of the target image by the second image segmentation model; and obtaining a target segmentation result of the target image based on a coincidence result between the first segmentation result and the second segmentation result.
[0005] In an embodiment of the present disclosure, the obtaining a target segmentation result of the target image based on a coincidence result between the first segmentation result and the second segmentation result includes: obtaining a first segmentation region of the target image in the first segmentation result; obtaining a second segmentation region of the target image in the second segmentation result; obtaining a coincidence region between the first segmentation region and the second segmentation region as a target segmentation region of the target image; and using the target segmentation region of the target image as the target segmentation result.
[0006] In one embodiment of the present disclosure, obtaining the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result includes: obtaining a first pixel point set of the target image in the first segmentation result, where the first category of each pixel point in the first pixel point set is a set category; obtaining a second pixel point set of the target image in the second segmentation result, where the second category of each pixel point in the second pixel point set is the set category; obtaining the intersection between the first pixel point set and the second pixel point set as the target pixel point set of the target image; and obtaining the target segmentation result based on the target pixel point set of the target image.
[0007] In one embodiment of the present disclosure, obtaining the target segmentation result based on the target pixel point set of the target image includes: generating a target segmentation region of the target image based on the target pixel point set of the target image; and using the target segmentation region of the target image as the target segmentation result.
[0008] In one embodiment of the present disclosure, obtaining the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result includes: obtaining the first category of the pixel points of the target image in the first segmentation result; obtaining the second category of the pixel points of the target image in the second segmentation result; if the first category of the pixel point is the same as the second category of the pixel point, using the first category as the target category of the pixel point; and obtaining the target segmentation result based on the target categories of multiple pixel points of the target image.
[0009] In one embodiment of the present disclosure, obtaining the target segmentation result based on the target categories of multiple pixel points of the target image includes: if the target category of the pixel point of the target image is the set category, adding the pixel point of the target image to the target pixel point set of the target image; generating a target segmentation region of the target image based on the target pixel point set of the target image; and using the target segmentation region of the target image as the target segmentation result.
[0010] In one embodiment of the present disclosure, the method further includes: if the first category of the pixel point is not the same as the second category of the pixel point, using a non-set category as the target category of the pixel point.
[0011] In one embodiment of the present disclosure, at least one of the first image segmentation model and the second image segmentation model is a saliency segmentation model.
[0012] According to a second aspect of the embodiments of the present disclosure, there is provided an image segmentation device, including: a first acquisition module configured to acquire a target image; a first segmentation module configured to input the target image into a first image segmentation model and output a first segmentation result of the target image by the first image segmentation model; a second segmentation module configured to input the target image into a second image segmentation model and output a second segmentation result of the target image by the second image segmentation model; a second acquisition module configured to obtain a target segmentation result of the target image based on a coincidence result between the first segmentation result and the second segmentation result.
[0013] In an embodiment of the present disclosure, the second acquisition module is configured to: acquire a first segmentation region of the target image in the first segmentation result; acquire a second segmentation region of the target image in the second segmentation result; acquire a coincidence region between the first segmentation region and the second segmentation region as a target segmentation region of the target image; and use the target segmentation region of the target image as the target segmentation result.
[0014] In an embodiment of the present disclosure, the second acquisition module is configured to: acquire a first pixel point set of the target image in the first segmentation result, where a first category of each pixel point in the first pixel point set is a set category; acquire a second pixel point set of the target image in the second segmentation result, where a second category of each pixel point in the second pixel point set is the set category; acquire an intersection between the first pixel point set and the second pixel point set as a target pixel point set of the target image; and obtain the target segmentation result based on the target pixel point set of the target image.
[0015] In an embodiment of the present disclosure, the second acquisition module is configured to: generate a target segmentation region of the target image based on the target pixel point set of the target image; and use the target segmentation region of the target image as the target segmentation result.
[0016] In an embodiment of the present disclosure, the second acquisition module is configured to: acquire a first category of a pixel point of the target image in the first segmentation result; acquire a second category of a pixel point of the target image in the second segmentation result; if the first category of the pixel point is the same as the second category of the pixel point, use the first category as the target category of the pixel point; and obtain the target segmentation result based on the target categories of multiple pixel points of the target image.
[0017] In one embodiment of the present disclosure, the second acquisition module is configured to perform: if the target category of the pixel point of the target image is a set category, add the pixel point of the target image to the target pixel point set of the target image; generate a target segmentation region of the target image based on the target pixel point set of the target image; and use the target segmentation region of the target image as the target segmentation result.
[0018] In one embodiment of the present disclosure, the second acquisition module is configured to perform: if the first category of the pixel point is inconsistent with the second category of the pixel point, use the non-set category as the target category of the pixel point.
[0019] In one embodiment of the present disclosure, at least one of the first image segmentation model and the second image segmentation model is a saliency segmentation model.
[0020] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including a processor; and a memory for storing processor-executable instructions; wherein, the processor is configured to implement the steps of the method according to the first aspect of the embodiments of the present disclosure.
[0021] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the method according to the first aspect of the embodiments of the present disclosure are implemented.
[0022] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, characterized in that when the computer program is executed by a processor of an electronic device, the steps of the method according to the first aspect of the embodiments of the present disclosure are implemented.
[0023] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects: The target image can be segmented by the first image segmentation model to obtain a first segmentation result, and the target image can be segmented by the second image segmentation model to obtain a second segmentation result. Considering the overlapping result between the first segmentation result and the second segmentation result, a target segmentation result is obtained. The target segmentation result can take into account the image segmentation performances of both image segmentation models, and can avoid the problem of inaccurate image segmentation caused by relying only on the image segmentation result of one image segmentation model, thereby improving the accuracy of image segmentation.
[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings herein are incorporated into and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an undue limitation of the present disclosure.
[0026] Figure 1 is a flowchart of an image segmentation method shown according to an exemplary embodiment.
[0027] Figure 2 is a flowchart of an image segmentation method shown according to another exemplary embodiment.
[0028] Figure 3 is a flowchart of an image segmentation method shown according to another exemplary embodiment.
[0029] Figure 4 is a flowchart of an image segmentation method shown according to another exemplary embodiment.
[0030] Figure 5 is a block diagram of an image segmentation apparatus shown according to an exemplary embodiment.
[0031] Figure 6 is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners
[0032] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above accompanying drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such data used may be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order different from those illustrated or described herein. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0034] In the technical solutions of the present disclosure, the acquisition, storage, use, processing, etc. of data all comply with the provisions of relevant laws and regulations.
[0035] Figure 1 is a flowchart of an image segmentation method shown according to an exemplary embodiment. As Figure 1 shown, the image segmentation method of the embodiments of the present disclosure includes the following steps.
[0036] S101, acquire a target image.
[0037] It should be noted that the execution subject of the image segmentation method in the embodiments of the present disclosure is an electronic device, and the electronic device includes terminals, such as mobile phones, notebooks, desktop computers, vehicle-mounted terminals, smart home appliances, wearable devices, etc. Among them, the wearable device may include a wrist-worn device (such as a smart watch, a smart bracelet), a head-mounted device, a foot-worn device, etc. The image segmentation method in the embodiments of the present disclosure may be executed by the image segmentation device in the embodiments of the present disclosure, and the image segmentation device in the embodiments of the present disclosure may be configured in any electronic device to execute the image segmentation method in the embodiments of the present disclosure.
[0038] It should be noted that the target image is not overly limited. For example, it may include two-dimensional images, three-dimensional images, etc.
[0039] In one implementation, obtaining the target image includes receiving the target image sent by the client. For example, taking the execution subject of the image segmentation method as the server, the client may obtain the target image based on the operation information of the user operating the client (such as the text input by the user, the icon clicked, the voice information of the user, etc.), and send the target image to the server. Correspondingly, the server may receive the target image sent by the client.
[0040] In one implementation, obtaining the target image includes receiving an image segmentation request sent by the client and determining the target image based on the image segmentation request.
[0041] In some examples, the image segmentation request carries an identifier of the target image, and the image identified by the identifier of the target image may be used as the target image. It should be noted that the identifier is not overly limited. For example, it may include a name, a storage path, etc.
[0042] In some examples, the image segmentation request carries the target image, and the target image may be extracted from the conversation request.
[0043] S102, input the target image into the first image segmentation model, and the first image segmentation model outputs the first segmentation result of the target image.
[0044] S103, input the target image into the second image segmentation model, and the second image segmentation model outputs the second segmentation result of the target image.
[0045] It should be noted that both the first image segmentation model and the second image segmentation model can be implemented by using any image segmentation model in the related art, and no excessive limitation is imposed here. For example, at least one of the first image segmentation model and the second image segmentation model is a saliency segmentation model. For example, the first image segmentation model and the second image segmentation model are image segmentation models of the same category or different categories.
[0046] In one embodiment, the first image segmentation model is a DenseCLIP (Language-Guided DensePrediction with Context-Aware Prompting) model, and the second image segmentation model is a U2net model. The DenseCLIP model has good generalization, and the U2net model has the advantages of smooth image segmentation edges and high accuracy.
[0047] In one embodiment, the inference process of the first image segmentation model is parallel to the inference process of the second image segmentation model.
[0048] It should be noted that neither the first segmentation result nor the second segmentation result of the image segmentation results is overly limited. For example, the first segmentation result may include the first segmentation region of the target image, the first pixel point set, the first category of the pixel points of the target image, the first probability of the pixel points of the target image under each candidate category, etc. The second segmentation result may include the second segmentation region of the target image, the second pixel point set, the second category of the pixel points of the target image, the second probability of the pixel points of the target image under each candidate category, etc.
[0049] Among them, the first segmentation region refers to the region of the set object in the target image. The first segmentation region is a partial region of the target image, and the set object is not overly limited. For example, it may include commodities, human faces, human bodies, buildings, etc. For example, taking the target image as a commodity image, the set object may include commodities.
[0050] Among them, the first pixel point set refers to the set composed of the pixel points of the set category in the target image, that is, the first category of each pixel point in the first pixel point set is the set category. The first pixel point set is a subset of the total pixel point set of the target image, and the total pixel point set refers to the set composed of each pixel point in the target image. The set category is not overly limited. For example, it may include commodities, human faces, human bodies, buildings, etc. For example, taking the target image as a commodity image, the set category may include commodities.
[0051] Among them, the first category of the pixel points of the target image is not overly limited. For example, it may include commodities, human faces, human bodies, buildings, backgrounds, etc. Different pixel points may correspond to the same first category or different first categories.
[0052] Among them, the first probabilities of the pixel points of the target image under different candidate categories may be the same or different.
[0053] It should be noted that the second segmentation region of the target image, the second pixel point set, the second category of the pixel points of the target image, and the second probability of the pixel points of the target image under each candidate category can respectively refer to the relevant contents of the first segmentation region of the target image, the first pixel point set, the first category of the pixel points of the target image, and the first probability of the pixel points of the target image under each candidate category in the above embodiments, which will not be elaborated here.
[0054] In one implementation manner, the method further includes obtaining the sample image and the sample segmentation result of the sample image, inputting the sample image into the model to be trained, and outputting the predicted segmentation result of the sample image by the model to be trained, and training the model to be trained based on the predicted segmentation result and the sample segmentation result. Among them, the model to be trained is the first image segmentation model or the second image segmentation model.
[0055] It should be noted that the sample segmentation result can refer to the relevant contents of the first segmentation result and the second segmentation result in the above embodiments. Training the model to be trained based on the predicted segmentation result and the sample segmentation result can be implemented by any model training method in the related technologies, which will not be elaborated here.
[0056] In some examples, training the model to be trained based on the predicted segmentation result and the sample segmentation result includes obtaining the loss function of the pre-trained model based on the predicted segmentation result and the sample segmentation result, and training the pre-trained model based on the loss function. It should be noted that the loss function is not overly limited. For example, it may include CE (Cross Entropy), MSE (Mean-Square Error), KL (Kullback-Leibler) divergence, contrast loss function, etc.
[0057] S104, obtaining the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result.
[0058] In the embodiments of the present disclosure, there is an overlapping part between the first segmentation result and the second segmentation result, that is, the coincidence result. That is to say, the coincidence result exists in both the first segmentation result and the second segmentation result. It can be understood that the coincidence result is part or all of the first segmentation result and the second segmentation result, and there may also be a different part between the first segmentation result and the second segmentation result, that is, the difference result.
[0059] It should be noted that the target segmentation result is not overly limited. For example, the target segmentation result may include the target segmentation region of the target image, the target pixel point set, the target category of the pixel points of the target image, the target probability of the pixel points of the target image under each candidate category, etc. The target segmentation result can refer to the relevant content of the first segmentation result and the second segmentation result in the above embodiments, which will not be elaborated here.
[0060] In one implementation manner, based on the coincidence result between the first segmentation result and the second segmentation result, the target segmentation result of the target image is obtained, including taking the coincidence result as the target segmentation result.
[0061] In one implementation manner, if the category of the coincidence result is a non-region category, based on the coincidence result between the first segmentation result and the second segmentation result, the target segmentation result of the target image is obtained, including generating the target segmentation region of the target image based on the coincidence result, and taking the target segmentation region of the target region as the target segmentation result.
[0062] In one implementation manner, the first image segmentation model is the DenseCLIP model, and the second image segmentation model is the U2net model. The DenseCLIP model has good generalization performance, and the U2net model has the advantages of smooth image segmentation edges and high accuracy. Therefore, the target segmentation result can combine the image segmentation performances of the DenseCLIP model and the U2net model, and has the advantages of smooth image segmentation edges and high accuracy.
[0063] The image segmentation method provided by the embodiments of the present disclosure obtains a target image, inputs the target image into a first image segmentation model, outputs the first segmentation result of the target image by the first image segmentation model, inputs the target image into a second image segmentation model, outputs the second segmentation result of the target image by the second image segmentation model, and obtains the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result. Thus, the target image can be segmented by the first image segmentation model to obtain the first segmentation result, and the target image can be segmented by the second image segmentation model to obtain the second segmentation result. Considering the coincidence result between the first segmentation result and the second segmentation result, the target segmentation result is obtained. The target segmentation result can combine the image segmentation performances of the two image segmentation models, and can avoid the problem of inaccurate image segmentation caused by relying only on the image segmentation result of one image segmentation model, thereby improving the accuracy of image segmentation.
[0064] Figure 2 is a flowchart of an image segmentation method shown according to another exemplary embodiment. As Figure 2 shown, the image segmentation method of the embodiments of the present disclosure includes the following steps.
[0065] S201, Obtain the target image.
[0066] S202, Input the target image into the first image segmentation model, and the first image segmentation model outputs the first segmentation result of the target image.
[0067] S203, Input the target image into the second image segmentation model, and the second image segmentation model outputs the second segmentation result of the target image.
[0068] For the relevant content of steps S201 - S203, refer to the above embodiments, which will not be elaborated here.
[0069] S204, Obtain the first segmentation region of the target image in the first segmentation result.
[0070] S205, Obtain the second segmentation region of the target image in the second segmentation result.
[0071] S206, Obtain the overlapping region between the first segmentation region and the second segmentation region as the target segmentation region of the target image.
[0072] S207, Use the target segmentation region of the target image as the target segmentation result.
[0073] In the embodiments of the present disclosure, the first segmentation result includes the first segmentation region of the target image, the second segmentation result includes the second segmentation region of the target image, and there is an overlapping part between the first segmentation region and the second segmentation region, that is, the overlapping region. The overlapping region is part or all of the first segmentation region and the second segmentation region. At this time, the overlapping region and the target segmentation region are both overlapping results.
[0074] The image segmentation method provided by the embodiments of the present disclosure obtains the first segmentation region of the target image in the first segmentation result, obtains the second segmentation region of the target image in the second segmentation result, obtains the overlapping region between the first segmentation region and the second segmentation region as the target segmentation region of the target image, and uses the target segmentation region of the target image as the target segmentation result. Thus, the overlapping region between the first segmentation region and the second segmentation region can be considered as the target segmentation region to obtain the target segmentation result, improving the accuracy of the target segmentation region and the accuracy of image segmentation.
[0075] Figure 3 is a flowchart of an image segmentation method shown according to another exemplary embodiment. As Figure 3 shown, the image segmentation method of the embodiments of the present disclosure includes the following steps.
[0076] S301, Obtain the target image.
[0077] S302. Input the target image into the first image segmentation model, and the first image segmentation model outputs the first segmentation result of the target image.
[0078] S303. Input the target image into the second image segmentation model, and the second image segmentation model outputs the second segmentation result of the target image.
[0079] For the relevant content of steps S301 - S303, reference can be made to the above embodiments, which will not be elaborated here.
[0080] S304. Obtain the first pixel point set of the target image in the first segmentation result, where the first category of each pixel point in the first pixel point set is the set category.
[0081] S305. Obtain the second pixel point set of the target image in the second segmentation result, where the second category of each pixel point in the second pixel point set is the set category.
[0082] S306. Obtain the intersection between the first pixel point set and the second pixel point set as the target pixel point set of the target image.
[0083] S307. Obtain the target segmentation result based on the target pixel point set of the target image.
[0084] In the embodiments of the present disclosure, the first segmentation result includes the first pixel point set of the target image, the second segmentation result includes the second pixel point set of the target image, there is an overlapping part, that is, an intersection, between the first pixel point set and the second pixel point set, and the intersection is used as the target pixel point set. That is to say, the target pixel point set is the intersection between the first pixel point set and the second pixel point set. At this time, the intersection between the first pixel point set and the second pixel point set and the target pixel point set are both overlapping results. The target category of each pixel point in the target pixel point set is the set category.
[0085] In one implementation, obtaining the target segmentation result based on the target pixel point set of the target image includes using the target pixel point set of the target image as the target segmentation result.
[0086] In one implementation, obtaining the target segmentation result based on the target pixel point set of the target image includes generating the target segmentation region of the target image based on the target pixel point set of the target image, and using the target segmentation region of the target image as the target segmentation result. For example, the region formed by each pixel point in the target pixel point set of the target image can be obtained as the target segmentation region of the target image.
[0087] For example, taking the target image as a product image and the set category as a product, the first pixel point set includes pixel points 1 to 10 of the target image, and the first category of pixel points 1 to 10 is a product. The second pixel point set includes pixel points 5 to 20 of the target image, and the second category of pixel points 5 to 20 is a product. Then, the intersection between the first pixel point set and the second pixel point set includes pixel points 5 to 10. Taking the intersection as the target pixel point set, that is, the target pixel point set includes pixel points 5 to 10, and the target category of pixel points 5 to 10 is a product.
[0088] For example, the target pixel point set of the target image can be used as the target segmentation result.
[0089] For example, the area composed of pixel points 5 to 10 in the target pixel point set of the target image can be obtained as the target segmentation area of the target image, and the target segmentation area of the target image can be used as the target segmentation result.
[0090] The image segmentation method provided by the embodiments of the present disclosure obtains the first pixel point set of the target image in the first segmentation result, where the first category of each pixel point in the first pixel point set is the set category, obtains the second pixel point set of the target image in the second segmentation result, where the second category of each pixel point in the second pixel point set is the set category, obtains the intersection between the first pixel point set and the second pixel point set as the target pixel point set of the target image, and obtains the target segmentation result based on the target pixel point set of the target image. Thus, the intersection between the first pixel point set and the second pixel point set can be considered as the target pixel point set to obtain the target segmentation result, improving the accuracy of the target pixel point set and the accuracy of image segmentation.
[0091] Figure 4 is a flowchart of an image segmentation method shown according to another exemplary embodiment, as Figure 4 shown, the image segmentation method of the embodiments of the present disclosure includes the following steps.
[0092] S401, obtain a target image.
[0093] S402, input the target image into a first image segmentation model, and output the first segmentation result of the target image by the first image segmentation model.
[0094] S403, input the target image into a second image segmentation model, and output the second segmentation result of the target image by the second image segmentation model.
[0095] For the relevant content of steps S401 - S403, reference can be made to the above embodiments, and details are not described herein again.
[0096] S404, obtain the first category of the pixel points of the target image in the first segmentation result.
[0097] S405. Obtain the second category of the pixel points of the target image in the second segmentation result.
[0098] S406. If the first category of the pixel point is the same as the second category of the pixel point, use the first category as the target category of the pixel point.
[0099] S407. Obtain the target segmentation result based on the target categories of multiple pixel points of the target image.
[0100] In an embodiment of the present disclosure, the first segmentation result includes the first category of the pixel points of the target image, and the second segmentation result includes the second category of the pixel points of the target image. If the first category of the pixel point is the same as the second category of the pixel point, that is, the first category and the second category of the pixel point coincide, use the first category as the target category of the pixel point. That is to say, if the first category of the pixel point is the same as the second category of the pixel point, the first category, the second category, and the target category of the pixel point are the same. At this time, the first category, the second category, and the target category of the pixel point are all coincidence results.
[0101] In one implementation manner, obtaining the target segmentation result based on the target categories of multiple pixel points of the target image includes using the target categories of multiple pixel points of the target image as the target segmentation result.
[0102] In one implementation manner, obtaining the target segmentation result based on the target categories of multiple pixel points of the target image includes: if the target category of the pixel point of the target image is a set category, add the pixel point of the target image to the target pixel point set of the target image, and use the target pixel point set of the target image as the target segmentation result.
[0103] In one implementation manner, obtaining the target segmentation result based on the target categories of multiple pixel points of the target image includes: if the target category of the pixel point of the target image is a set category, add the pixel point of the target image to the target pixel point set of the target image, generate the target segmentation region of the target image based on the target pixel point set of the target image, and use the target segmentation region of the target image as the target segmentation result.
[0104] In one implementation manner, the method further includes: if the first category of the pixel point is different from the second category of the pixel point, use a non-set category as the target category of the pixel point. There is no excessive limitation on the non-set category. For example, taking the target image as a commodity image and the set category as a commodity, the non-set category may include background, face, human body, etc.
[0105] For example, taking the target image as a product image and the set category as a product, the first category of pixel points 1 to 10 is a product, the first category of pixel points 11 to 20 is a background, the second category of pixel points 1 to 4 is a background, and the second category of pixel points 5 to 20 is a product. It can be seen that the first category of pixel point i is the same as the second category of pixel point i, where i is a positive integer and 5 ≤ i ≤ 10. The product can be used as the target category of pixel point i, that is, the target category of pixel points 5 to 10 is a product.
[0106] For example, the target category of pixel points 5 to 10 being a product can be used as the target segmentation result.
[0107] For example, pixel points 5 to 10 can be added to the set of target pixel points of the target image, and the set of target pixel points of the target image can be used as the target segmentation result.
[0108] For example, the area composed of pixel points 5 to 10 in the set of target pixel points of the target image can be obtained as the target segmentation area of the target image, and the target segmentation area of the target image can be used as the target segmentation result.
[0109] For example, it can be seen that the first category of pixel point j is the same as the second category of pixel point j, where j is a positive integer and the value range of j includes 1 to 4 and 11 to 20. The background can be used as the target category of pixel point j, that is, the target category of pixel points 1 to 4 and 11 to 20 is a background.
[0110] The image segmentation method provided by the embodiments of the present disclosure obtains the first category of the pixel points of the target image in the first segmentation result, obtains the second category of the pixel points of the target image in the second segmentation result. If the first category of the pixel point is the same as the second category of the pixel point, the first category is used as the target category of the pixel point. Based on the target categories of multiple pixel points of the target image, the target segmentation result is obtained. Thus, if the first category of the pixel point is the same as the second category of the pixel point, the first category is used as the target category of the pixel point to obtain the target segmentation result, improving the accuracy of the target category of the pixel point and the accuracy of image segmentation.
[0111] Figure 5 is a block diagram of an image segmentation device shown according to an exemplary embodiment. Refer to Figure 5 , the image segmentation device 100 of the embodiments of the present disclosure includes: a first acquisition module 110, a first segmentation module 120, a second segmentation module 130, and a second acquisition module 140.
[0112] The first acquisition module 110 is configured to acquire a target image;
[0113] The first segmentation module 120 is configured to input the target image into a first image segmentation model, and output a first segmentation result of the target image by the first image segmentation model;
[0114] The second segmentation module 130 is configured to input the target image into a second image segmentation model, and output a second segmentation result of the target image by the second image segmentation model;
[0115] The second acquisition module 140 is configured to obtain a target segmentation result of the target image based on an overlapping result between the first segmentation result and the second segmentation result.
[0116] In an embodiment of the present disclosure, the second acquisition module 140 is configured to: acquire a first segmentation region of the target image in the first segmentation result; acquire a second segmentation region of the target image in the second segmentation result; acquire an overlapping region between the first segmentation region and the second segmentation region as a target segmentation region of the target image; and use the target segmentation region of the target image as the target segmentation result.
[0117] In an embodiment of the present disclosure, the second acquisition module 140 is configured to: acquire a first pixel point set of the target image in the first segmentation result, where a first category of each pixel point in the first pixel point set is a set category; acquire a second pixel point set of the target image in the second segmentation result, where a second category of each pixel point in the second pixel point set is the set category; acquire an intersection between the first pixel point set and the second pixel point set as a target pixel point set of the target image; and obtain the target segmentation result based on the target pixel point set of the target image.
[0118] In an embodiment of the present disclosure, the second acquisition module 140 is configured to: generate a target segmentation region of the target image based on the target pixel point set of the target image; and use the target segmentation region of the target image as the target segmentation result.
[0119] In an embodiment of the present disclosure, the second acquisition module 140 is configured to: acquire a first category of a pixel point of the target image in the first segmentation result; acquire a second category of a pixel point of the target image in the second segmentation result; if the first category of the pixel point is the same as the second category of the pixel point, use the first category as a target category of the pixel point; and obtain the target segmentation result based on the target categories of multiple pixel points of the target image.
[0120] In one embodiment of the present disclosure, the second acquisition module 140 is configured to perform: if the target category of the pixel point of the target image is a set category, add the pixel point of the target image to the target pixel point set of the target image; generate a target segmentation region of the target image based on the target pixel point set of the target image; and use the target segmentation region of the target image as the target segmentation result.
[0121] In one embodiment of the present disclosure, the second acquisition module 140 is configured to perform: if the first category of the pixel point is inconsistent with the second category of the pixel point, use a non-set category as the target category of the pixel point.
[0122] In one embodiment of the present disclosure, at least one of the first image segmentation model and the second image segmentation model is a saliency segmentation model.
[0123] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.
[0124] The image segmentation device provided by the embodiments of the present disclosure acquires a target image, inputs the target image into a first image segmentation model, and the first image segmentation model outputs a first segmentation result of the target image. Then, the target image is input into a second image segmentation model, and the second image segmentation model outputs a second segmentation result of the target image. Based on the coincidence result between the first segmentation result and the second segmentation result, a target segmentation result of the target image is obtained. Thus, the target image can be segmented by the first image segmentation model to obtain a first segmentation result, and the target image can be segmented by the second image segmentation model to obtain a second segmentation result. Considering the coincidence result between the first segmentation result and the second segmentation result, the target segmentation result can incorporate the image segmentation performances of the two image segmentation models, and can avoid the problem of inaccurate image segmentation caused by relying only on the image segmentation result of one image segmentation model, thereby improving the accuracy of image segmentation.
[0125] Figure 6 It is a block diagram of an electronic device shown according to an exemplary embodiment.
[0126] As Figure 6 shown, the above electronic device 200 includes:
[0127] A memory 210, a processor 220, and a bus 230 connecting different components (including the memory 210 and the processor 220). The memory 210 stores a computer program, and when the processor 220 executes the program, the image segmentation method described in the embodiments of the present disclosure is implemented.
[0128] Bus 230 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of the several bus architectures. By way of example, and not limitation, such architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0129] Electronic device 200 typically includes a variety of electronic device-readable media. These media can be any available media that can be accessed by electronic device 200, including both volatile and nonvolatile media, removable and non-removable media.
[0130] Memory 210 may also include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. Electronic device 200 may further include other removable / non-removable, volatile / nonvolatile computer system storage media. By way of example only, storage system 260 can be used for reading from and writing to non-removable, nonvolatile magnetic media ( Figure 6 not shown and typically called a "hard disk drive"). Although Figure 6 not shown in the figures, a disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from and writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) can be provided. In these instances, each drive can be connected to bus 230 by one or more data media interfaces. Memory 210 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the present disclosure.
[0131] A program / utility 280 having a set (at least one) of program modules 270 can be stored, for example, in memory 210, such program modules 270 including—but not limited to—an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may include an implementation of a networking environment. The program modules 270 generally carry out the functions and / or methods of the embodiments described herein.
[0132] The electronic device 200 can also communicate with one or more external devices 290 (such as a keyboard, a pointing device, a display 291, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 200, and / or communicate with any device that enables the electronic device 200 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 292. Moreover, the electronic device 200 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 293. As Figure 6 shown, the network adapter 293 communicates with other modules of the electronic device 200 through a bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0133] The processor 220 executes various functional applications and data processing by running programs stored in the memory 210.
[0134] It should be noted that for the implementation process and technical principle of the electronic device in this embodiment, refer to the foregoing explanation of the image segmentation method of the embodiments of the present disclosure, and details will not be elaborated here.
[0135] The electronic device provided by the embodiments of the present disclosure can execute the image segmentation method as described above, obtain a target image, input the target image into a first image segmentation model, output a first segmentation result of the target image by the first image segmentation model, input the target image into a second image segmentation model, output a second segmentation result of the target image by the second image segmentation model, and obtain a target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result. Thus, the target image can be segmented by the first image segmentation model to obtain a first segmentation result, and the target image can be segmented by the second image segmentation model to obtain a second segmentation result. Considering the coincidence result between the first segmentation result and the second segmentation result, a target segmentation result is obtained. The target segmentation result can incorporate the image segmentation performance of both image segmentation models, and can avoid the problem of inaccurate image segmentation caused by relying solely on the image segmentation result of one image segmentation model, thereby improving the accuracy of image segmentation.
[0136] To implement the above embodiments, the present disclosure also proposes a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the image segmentation method provided by the present disclosure are implemented.
[0137] Optionally, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0138] To implement the above embodiments, the present disclosure also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor of an electronic device, the image segmentation method described above is implemented.
[0139] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.
[0140] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.
Claims
1. An image segmentation method, characterized in that, Including: Obtain a target image; Input the target image into a first image segmentation model, and output a first segmentation result of the target image by the first image segmentation model; Input the target image into a second image segmentation model, and output a second segmentation result of the target image by the second image segmentation model; Based on the coincidence result between the first segmentation result and the second segmentation result, obtain a target segmentation result of the target image.
2. The method according to claim 1, characterized in that, The obtaining the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result includes: Obtain a first segmentation region of the target image in the first segmentation result; Obtain a second segmentation region of the target image in the second segmentation result; Obtain a coincidence region between the first segmentation region and the second segmentation region as a target segmentation region of the target image; Use the target segmentation region of the target image as the target segmentation result.
3. The method according to claim 1, characterized in that, The obtaining the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result includes: Obtain a first pixel point set of the target image in the first segmentation result, where the first category of each pixel point in the first pixel point set is a set category; Obtain a second pixel point set of the target image in the second segmentation result, where the second category of each pixel point in the second pixel point set is the set category; Obtain an intersection between the first pixel point set and the second pixel point set as a target pixel point set of the target image; Based on the target pixel point set of the target image, obtain the target segmentation result.
4. The method according to claim 3, characterized in that, The obtaining the target segmentation result based on the target pixel point set of the target image includes: Generate a target segmentation region of the target image based on the target pixel point set of the target image; Use the target segmentation region of the target image as the target segmentation result.
5. The method according to claim 1, characterized in that, The obtaining the target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result includes: Obtain a first category of a pixel point of the target image in the first segmentation result; Obtain a second category of a pixel point of the target image in the second segmentation result; If the first category of the pixel point is the same as the second category of the pixel point, use the first category as the target category of the pixel point; Based on the target categories of multiple pixel points of the target image, obtain the target segmentation result.
6. The method according to claim 5, characterized in that, The obtaining the target segmentation result based on the target categories of multiple pixel points of the target image includes: If the target category of a pixel point of the target image is a set category, add the pixel point of the target image to the target pixel point set of the target image; Generate a target segmentation region of the target image based on the target pixel point set of the target image; Use the target segmentation region of the target image as the target segmentation result.
7. The method according to claim 5, characterized in that, The method further includes: If the first category of the pixel is inconsistent with the second category of the pixel, the non-set category is used as the target category of the pixel.
8. The method according to any one of claims 1-7, characterized in that, At least one of the first image segmentation model and the second image segmentation model is a saliency segmentation model.
9. An image segmentation device, characterized in that, Including: A first acquisition module configured to acquire a target image; A first segmentation module configured to input the target image into a first image segmentation model, and output a first segmentation result of the target image by the first image segmentation model; A second segmentation module configured to input the target image into a second image segmentation model, and output a second segmentation result of the target image by the second image segmentation model; A second acquisition module configured to obtain a target segmentation result of the target image based on the coincidence result between the first segmentation result and the second segmentation result.
10. An electronic device, characterized in that, Including: A processor; A memory for storing processor-executable instructions; Wherein, the processor is configured to: Implement the steps of the method according to any one of claims 1-8.
11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instruction is executed by the processor, the steps of the method according to any one of claims 1-8 are implemented.