Image segmentation method, image segmentation device, electronic equipment and storage medium

Abstract

The invention relates to an image segmentation method, an image segmentation device, electronic equipment and a storage medium. The image segmentation method comprises the following steps: acquiring a to-be-processed image, and determining a to-be-segmented target in the to-be-processed image; and determining a template image according to the target to be segmented. And determining a first matching point in the to-be-processed image according to the image features of the to-be-segmented target image. According to the first matching point, identification information is determined, and the identification information is used for identifying an image area corresponding to a to-be-segmented target in the to-be-processed image. And according to the identification information, performing segmentation in the to-be-processed image to obtain the to-be-segmented target. According to the image segmentation method and device, the identification information used for segmenting the to-be-segmented target in the to-be-processed image is determined according to the template image containing the to-be-segmented target image, a user is prevented from setting the identification information by himself / herself for the to-be-segmented target, inconvenience brought by manual setting of the identification information is avoided, and the image segmentation efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image segmentation method, an image segmentation device, an electronic device, and a storage medium. Background Art

[0002] Image segmentation is a technology that divides an image into multiple image regions with different semantics or features. Image segmentation has wide applications in computer vision, medical imaging, remote sensing and other fields. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides an image segmentation method, an image segmentation device, an electronic device and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image segmentation method, comprising: acquiring an image to be processed, and determining a target to be segmented in the image to be processed; determining a template image based on the target to be segmented, wherein the template image is a pre-set image and the template image includes the target image to be segmented; determining a first matching point in the image to be processed based on image features of the target image to be segmented, wherein the first matching point is a pixel point in the image to be processed corresponding to the target image to be segmented; determining identification information based on the first matching point, wherein the identification information is used to identify an image area corresponding to the target to be segmented in the image to be processed; and segmenting the target to be segmented in the image to be processed based on the identification information.

[0005] In one embodiment, determining the first matching point in the image to be processed based on the image features of the target image to be segmented includes: performing image scale conversion on the image to be processed according to multiple preset image sizes to obtain multiple converted images corresponding to the multiple preset image sizes; determining the second matching point corresponding to the target image to be segmented in each image included in an image set based on the image features of the target image to be segmented, the image set including the image to be processed and the multiple converted images, the second matching point being a pixel point in the image corresponding to the target to be segmented in each image included in the image set; and determining the first matching point based on the second matching point.

[0006] In one embodiment, the method further includes: after obtaining the multiple converted images, respectively saving the pixel coordinate conversion relationship between the image to be processed and each converted image; determining the first matching point based on the second matching point includes: mapping the second matching point in each converted image to the image to be processed based on the pixel coordinate conversion relationship between each converted image in the multiple converted images and the image to be processed, and determining the second matching point mapped to the image to be processed as the mapping matching point; and determining the union of the second matching point in the image to be processed and the mapping matching point as the first matching point.

[0007] In one embodiment, determining the identification information based on the first matching points includes: clustering the first matching points by density clustering according to a preset number and a preset distance to obtain a plurality of first clusters composed of first matching points, wherein the number of first matching points in the first cluster is greater than or equal to the preset number, and the neighborhood radius of the first cluster is the preset distance; determining the identification information based on the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed.

[0008] In one embodiment, the identification information includes an identification point corresponding to the target to be segmented and an identification frame including the target to be segmented; the identification information is determined based on the distribution of the first matching points in the multiple first clusters and the distribution of the multiple first clusters in the image to be processed, including: determining the first cluster containing the most first matching points among the multiple first clusters as the second cluster, and determining multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster, and determining the center coordinates of the multiple pixel coordinates as the identification point; determining a minimum frame surrounding the multiple first clusters, and determining the minimum frame as the identification frame.

[0009] In one embodiment, segmenting the target to be segmented in the image to be processed based on the identification information includes: inputting the image to be processed, the identification points and the identification box into an image segmentation model, and segmenting the image area identified by the identification points and the identification box in the image to be processed; the image segmentation model is used to segment the image to be processed based on the input identification points and identification boxes.

[0010] In one embodiment, the image segmentation model is a SAM (Segmentation Model).

[0011] According to a second aspect of an embodiment of the present disclosure, an image segmentation device is provided, comprising: a determination unit for acquiring an image to be processed and determining a target to be segmented in the image to be processed, and for determining a template image based on the target to be segmented, wherein the template image is a pre-set image and the template image includes the target image to be segmented; a processing unit for determining a first matching point in the image to be processed based on image features of the target image to be segmented, wherein the first matching point is a pixel point in the image to be processed corresponding to the target image to be segmented, and identification information is determined based on the first matching point, wherein the identification information is used to identify an image area corresponding to the target to be segmented in the image to be processed; and a segmentation unit for segmenting the target to be segmented in the image to be processed based on the identification information.

[0012] In one embodiment, the processing unit determines a first matching point in the image to be processed based on the image features of the target image to be segmented in the following manner: performing image scale conversion on the image to be processed according to multiple preset image sizes to obtain multiple converted images corresponding to the multiple preset image sizes; determining a second matching point corresponding to the target image to be segmented in each image included in an image set according to the image features of the target image to be segmented, the image set including the image to be processed and the multiple converted images, the second matching point being a pixel point in the image corresponding to the target to be segmented in each image included in the image set; determining the first matching point based on the second matching point.

[0013] In one embodiment, the processing unit is further used to: after obtaining the multiple conversion images, respectively save the pixel coordinate conversion relationship between the image to be processed and each conversion image; the processing unit determines the first matching point based on the second matching point in the following manner: according to the pixel coordinate conversion relationship between each conversion image in the multiple conversion images and the image to be processed, map the second matching point in each conversion image to the image to be processed, and determine the second matching point mapped to the image to be processed as the mapping matching point; determine the union of the second matching point in the image to be processed and the mapping matching point as the first matching point.

[0014] In one embodiment, the processing unit determines the identification information based on the first matching points in the following manner: clustering the first matching points by density clustering according to a preset number and a preset distance to obtain a plurality of first clusters composed of first matching points, wherein the number of first matching points in the first cluster is greater than or equal to the preset number, and the neighborhood radius of the first cluster is the preset distance; determining the identification information based on the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed.

[0015] In one embodiment, the identification information includes an identification point corresponding to the target to be segmented and an identification frame including the target to be segmented; the processing unit determines the identification information according to the distribution of the first matching points in the multiple first clusters and the distribution of the multiple first clusters in the image to be processed in the following manner: the first cluster containing the most first matching points among the multiple first clusters is determined as the second cluster, and the multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster are determined, and the center coordinates of the multiple pixel coordinates are determined as the identification point; the minimum frame surrounding the multiple first clusters is determined, and the minimum frame is determined as the identification frame.

[0016] In one embodiment, the segmentation unit segments the target to be segmented in the image to be processed according to the identification information pair in the following manner: the image to be processed, the identification point and the identification box are input into an image segmentation model, and the image area identified by the identification point and the identification box in the image to be processed is segmented; the image segmentation model is used to segment the image to be processed according to the input identification point and identification box.

[0017] In one embodiment, the image segmentation model is a SAM (Segmentation Model).

[0018] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to: execute the image segmentation method described in the first aspect or any one of the embodiments of the first aspect.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the image segmentation method described in the first aspect or any one of the embodiments of the first aspect.

[0020] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: after acquiring the image to be processed, a pre-set template image including the target image to be segmented is determined based on the target to be segmented in the image to be processed. Based on the image features of the target image to be segmented in the template image, a first matching point that matches the target to be segmented is determined in the image to be processed. Based on the first matching point, identification information for identifying the image area corresponding to the target to be segmented in the image to be processed is determined. The target to be segmented is obtained by segmenting in the image to be processed based on the identification information. Through the present disclosure, based on the template image including the target image to be segmented, identification information for segmenting the target to be segmented in the image to be processed is determined, thereby avoiding users from setting identification information for the target to be segmented by themselves, avoiding the inconvenience caused by manually setting identification information, and improving image segmentation efficiency.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] Figure 1 The figure is a schematic diagram showing image segmentation using an image segmentation model according to an exemplary embodiment of the present disclosure.

[0024] Figure 2 The figure is a flowchart of an image segmentation method according to an exemplary embodiment.

[0025] Figure 3 The present invention is a flowchart showing a method for determining a first matching point in an image to be processed according to an exemplary embodiment.

[0026] Figure 4 is a schematic diagram showing a pixel matching relationship according to an exemplary embodiment of the present disclosure.

[0027] Figure 5 The flowchart of a method for determining a first matching point according to a second matching point is shown according to an exemplary embodiment.

[0028] Figure 6 The figure is a flowchart showing a method for determining identification information according to a first matching point according to an exemplary embodiment.

[0029] Figure 7 The figure is a flowchart showing a method for determining identification information according to an exemplary embodiment.

[0030] Figure 8This is a flowchart showing a method for segmenting an image to be processed based on identification information according to an exemplary embodiment.

[0031] Figure 9 The figure is a flowchart of an image segmentation method according to an exemplary embodiment of the present disclosure.

[0032] Figure 10 is a schematic diagram showing an image segmentation processing result according to an exemplary embodiment of the present disclosure.

[0033] Figure 11 is a schematic diagram showing an image segmentation processing result according to an exemplary embodiment of the present disclosure.

[0034] Figure 12A 3 is a schematic diagram showing a processing result of segmenting an image to be segmented according to an exemplary embodiment of the present disclosure.

[0035] Figure 12B 3 is a schematic diagram showing a processing result of segmenting an image to be segmented according to an exemplary embodiment of the present disclosure.

[0036] Figure 12C 3 is a schematic diagram showing a processing result of segmenting an image to be segmented according to an exemplary embodiment of the present disclosure.

[0037] Figure 12D 3 is a schematic diagram showing a processing result of segmenting an image to be segmented according to an exemplary embodiment of the present disclosure.

[0038] Figure 13 The figure is a block diagram of an image segmentation apparatus according to an exemplary embodiment.

[0039] Figure 14 The figure is a block diagram of a device for image segmentation according to an exemplary embodiment. DETAILED DESCRIPTION

[0040] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure.

[0041] The image processing method provided by the embodiments of the present disclosure is applied to a scenario where an image to be processed is segmented.

[0042] Image segmentation technology is a technology that divides an image into multiple images with different semantics or features. Image segmentation technology has been widely used in computer vision, medical imaging, remote sensing and other fields.

[0043] In related technologies, with the development of deep learning and neural networks, the field of image segmentation has also made many new progress and breakthroughs, which can be roughly divided into the following stages: (1) Image segmentation based on the patch classification method, that is, cutting the image into blocks and feeding them into the deep model, and classifying the pixels through the deep model. Since the deep model corresponding to the patch classification method is a deep model based on the fully connected layer, the image segmentation method based on the patch classification method can only process images of fixed size. (2) Image segmentation method based on the fully convolutional neural network (FCN). The fully convolutional neural network replaces the fully connected layer of the network with the convolution layer, which can process images of any size and is much faster than the patch classification method. However, position information is very important in semantic segmentation (image segmentation based on semantics), and the downsampling operation of the convolutional neural network (CNN) model in the fully convolutional neural network will still reduce the image resolution, thereby weakening the position information and ultimately affecting the image segmentation processing results. (3) Image segmentation methods based on the encoder-decoder framework (such as the U-Net series), where the encoder gradually reduces the spatial dimension through downsampling, while the decoder gradually restores the spatial dimension and detail information through upsampling. The introduction of the dilated convolution structure can also make up for the shortcomings of the CNN downsampling operation, maintain spatial resolution, and expand the receptive field, thereby integrating contextual information and outputting it. In the post-processing of segmentation results, related technologies include post-processing based on conditional random fields (CRFs) to improve the segmentation effect. For example, the DeepLab semantic segmentation model uses a post-processing method based on conditional random fields to better improve the segmentation results.

[0044] Based on whether manual annotation of data is required, image segmentation methods can be divided into three categories: supervised, semi-supervised, and unsupervised. Both supervised and semi-supervised image segmentation methods require manual annotation of data, and combine manually annotated data for image segmentation, which not only brings inconvenience but also affects the processing efficiency of image segmentation. Most of the above methods require supervised training with the help of manually annotated data, and the usage scenarios are limited by the training data. They are supervised or semi-supervised image segmentation methods, and there are problems such as the need to manually set parameters (i.e., annotated data) and low processing efficiency. Therefore, there is also an unsupervised image segmentation method combined with computer vision (CV) in the related art, namely the SegmentAnything Model (SAM). The SAM model can segment any object in any image without any annotation, only by inputting the corresponding prompts. The SAM model can automatically complete the segmentation task without additional training. Such as Figure 1 The schematic diagram of image segmentation using the image segmentation model is shown in FIG. The SAM model specifically consists of three parts: image encoder (image-encoder), prompt encoder (prompt-encoder) and mask encoder (mask-encoder). The image encoder uses the Vision Transformer (ViT) structure to extract image features; the prompt encoder contains three forms of prompt input, namely points, boxes, and text. The prompt in point form is mapped using position encoding, the prompt of the box uses the coordinates of the upper left corner and the lower right corner of the box, and the two are also mapped using position encoding. The text prompt word is encoded using a text encoder based on the Contrastive Language-Image Pre-training (CLIP) model; after obtaining the embedding layer of the image to be processed (including image areas A, B, and C) and the prompt, the two are simultaneously sent to the mask encoder (the mask is sent in the form of a convolution kernel conv), and after the feature conversion of the transformer structure, the panoramic segmentation result of the image is finally output, namely the valid mask (valid The panoramic segmentation results include the segmentation result score1 of image regions A, B, and C with masks, the segmentation result score2 of image regions A and B with masks, and the segmentation result score3 of image region A with masks.

[0045] The above-mentioned SAM model is an unsupervised image segmentation method that can segment any object in the image without any annotation. However, the SAM model lacks certain semantic information and cannot know the category information of the segmented object: that is, because the SAM model performs panoramic segmentation on the image, it will segment each type of object contained in the image. However, if you want to obtain the segmentation result of a specific category, the user needs to manually enter the corresponding prompt (point, box or text prompt word) of the category to obtain the segmentation result corresponding to the specific category in a targeted manner. Manual prompt input for specific categories will limit the practicality of the model. For example, in large-scale image segmentation scenarios, manually entering prompts for each image to segment is too inefficient and too complex to meet the requirements.

[0046] In view of the problems existing in the above-mentioned SAM model, the present disclosure proposes an image segmentation method. After obtaining the image to be processed, a pre-set template image including the target image to be segmented is determined based on the target to be segmented in the image to be processed. According to the image features of the target image to be segmented in the template image, a first matching point that matches the target to be segmented is determined in the image to be processed. According to the first matching point, identification information for identifying the image area corresponding to the target to be segmented in the image to be processed is determined. The target to be segmented is obtained by segmenting in the image to be processed based on the identification information. Through the present disclosure, the identification information for segmenting the target to be segmented in the image to be processed is determined based on the template image containing the target image to be segmented, thereby avoiding the user from setting the identification information for the target to be segmented by themselves, avoiding the inconvenience caused by manually setting the identification information, and improving the efficiency of image segmentation.

[0047] The technical field to which this technical solution belongs is the field of computer vision-image segmentation, and it can also be extended to other technical fields.

[0048] Figure 2 FIG. 1 is a flow chart showing an image segmentation method according to an exemplary embodiment. Figure 2 As shown, the method includes steps S101 to S105.

[0049] In step S101 , an image to be processed is acquired, and an object to be segmented in the image to be processed is determined.

[0050] In step S102 , a template image is determined according to the target to be segmented. The template image is a preset image, and the template image includes the target image to be segmented.

[0051] In step S103, a first matching point is determined in the image to be processed according to the image features of the target image to be segmented. The first matching point is a pixel point in the image to be processed corresponding to the target image to be segmented.

[0052] In step S104, identification information is determined according to the first matching point, where the identification information is used to identify an image region corresponding to the object to be segmented in the image to be processed.

[0053] In step S105 , the target to be segmented is obtained by segmenting the image to be processed according to the identification information.

[0054] In an embodiment of the present disclosure, the target to be segmented in the image to be processed is an image corresponding to a specific target in the image to be processed, such as an image corresponding to a billboard in a street view image, or an image corresponding to a car in a street view image, or an image corresponding to a person in a video screenshot. The template image in the present disclosure is a pre-set image, and the template image also contains the image of the above-mentioned specific target. It can be understood that based on the need to perform image segmentation on a specific object, the present disclosure sets template images for various specific targets with image segmentation needs, so that when there is a need for image segmentation, the corresponding template image can be selected according to the specific target in the image to be processed. It can be further understood that although the image layout and shooting angle corresponding to the target to be segmented in the image to be processed may be different from the image layout and shooting angle corresponding to the target to be segmented in the template image, the pixel point corresponding to the target to be segmented (first matching point) can be determined in the image to be processed based on the image features corresponding to the target to be segmented in the template image, and then the identification information is determined according to the first matching point and the image segmentation is performed.

[0055] In an embodiment of the present disclosure, after obtaining the pixel points corresponding to the target image to be segmented in the image to be processed, identification information for identifying the target to be segmented is further determined based on the distribution of the pixel points in the image to be processed, and the image to be processed is segmented based on the identification information to obtain the target to be segmented.

[0056] Through the present disclosure, after acquiring an image to be processed, a target image is determined based on the target to be segmented in the image to be processed. Based on a template image containing the target image to be segmented, identification information for segmenting the target to be segmented in the image to be processed is determined, and image segmentation is performed based on the identification information. This eliminates the need for users to manually set identification information for the target to be segmented, avoiding the inconvenience of manually setting identification information and improving image segmentation efficiency. For example, in large-scale image segmentation scenarios, identification information can be obtained from the template image to automatically complete image segmentation, improving image segmentation efficiency.

[0057] It is understandable that the target to be segmented may be presented in a variety of forms in the image to be processed, and the corresponding pixel quantities may also be different, which will affect the feature matching between the image to be processed and the template image, and affect the acquisition of identification information, and ultimately affect the segmentation of the target to be segmented. In view of the impact of the presentation form of the target to be segmented on image segmentation, the present disclosure will perform image scale conversion on the image to be processed, obtain converted images corresponding to the image to be processed and having different image sizes, and then determine matching points in the image to be processed with different image sizes based on the features of the target image to be segmented in the template image. The following embodiments of the present disclosure illustrate the method for determining the first matching point in the image to be processed.

[0058] Figure 3 FIG. 1 is a flow chart showing a method for determining a first matching point in an image to be processed according to an exemplary embodiment. Figure 3 As shown, the method includes steps S201 to S203.

[0059] In step S201 , image scale conversion is performed on the image to be processed according to a plurality of preset image sizes, so as to obtain a plurality of converted images corresponding to the plurality of preset image sizes.

[0060] In step S202 , a second matching point corresponding to the target image to be segmented is determined in each image included in the image set based on the image features of the target image to be segmented.

[0061] The image set includes the image to be processed and the converted image, and the second matching point is a pixel point in the image corresponding to the target to be segmented in each image included in the image set.

[0062] In step S203, the first matching point is determined according to the second matching point.

[0063] It is understandable that due to the differences in shooting distance and shooting angle, the target to be segmented in the image to be processed will be presented in different forms, and will differ from the target to be segmented in the template image in terms of image size and presentation angle. For example, the target to be segmented in the close-up image is relatively obvious, and there are relatively more matching points with the template image, so it is easier to distinguish it from the background, and it is relatively easy to obtain the first matching point. However, the target in the distant image is relatively small, and the number of matching points is relatively small, which makes it easy to make matching errors, and it is relatively difficult to obtain the first matching point. In view of the impact of the above differences on obtaining the first matching point when performing feature matching. The present disclosure adopts a multi-scale matching method, that is, multi-scale changes are made to the image to be processed, and by scaling the original image size of the image to be processed to different scales, the effect of magnifying the target in the distant image is achieved, thereby accurately predicting the matching points, obtaining the first matching points that meet the requirements, and ensuring that accurate identification information is obtained based on the first matching points, thereby achieving accurate segmentation of the target to be segmented.

[0064] In the embodiment of the present disclosure, based on the image features of the image to be segmented in the template image, pixels with the same image features in the image to be processed are obtained. In one example, Figure 4 The schematic diagram of the pixel matching relationship is shown. Figure 4 Where A is the template image and B is the image to be processed. Based on the corresponding image features in A and B, matching pixel points corresponding to specific image features in A can be determined in B. The present disclosure pre-sets multiple different image sizes, performs size conversion on the image to be processed, and obtains a converted image at each size. Based on the above-mentioned pixel matching relationship based on image features, the image to be processed is feature-matched with the image template, and the converted images of multiple different sizes are respectively feature-matched with the image template, thereby obtaining matching pixel points (second matching points) in each image (i.e., the image to be processed and the multiple converted images).

[0065] Through the present disclosure, converted images corresponding to the image to be processed and having different image sizes are obtained, and based on the feature matching relationship between the template image and each converted image, and the feature matching relationship between the template image and the image to be processed, pixel points (i.e., second matching points) matching the template image are obtained from the image to be processed and the multiple converted images, and then the first matching points are obtained based on the second matching points, and identification information and image segmentation are obtained, to ensure that the number of first matching points obtained meets the requirements, and to ensure that accurate identification information is obtained based on the first matching points, thereby achieving accurate segmentation of the target to be segmented.

[0066] In an exemplary embodiment of the present disclosure, a multi-scale matching module is used to perform scale conversion of the image to be processed and obtain the second matching points. In order to ensure the fairness of the number of matching points in different test images, the present disclosure sets a fixed template image size, and only performs multi-scale changes on the original image to be processed. The original image is scaled based on three sizes of 840, 1024 and 1280, and then the LightGlue model is used to perform feature matching on the images of these four scales (original image size, 840, 1024 and 1280) and the template image, and outputs feature matching points (second matching points) of images of different scales.

[0067] It will be appreciated that the image segmentation method disclosed herein primarily targets the image to be processed. Therefore, the method requires mapping the second matching points obtained in each converted image to the image to be processed, and acquiring identification information based on the second matching points mapped to the image to be processed and the inherent second matching points in the image to be processed. The following embodiments of the present disclosure illustrate the method for determining the first matching points.

[0068] Figure 5 FIG. 1 is a flow chart showing a method for determining a first matching point based on a second matching point according to an exemplary embodiment. Figure 5As shown, the method includes steps S301 to S302.

[0069] In step S301, according to the pixel coordinate conversion relationship between each converted image and the image to be processed, the second matching point in each converted image is mapped to the image to be processed, and the second matching point mapped to the image to be processed is determined as the mapped matching point.

[0070] In step S302, the union of the second matching point and the mapped matching point in the image to be processed is determined as the first matching point.

[0071] In the disclosed embodiment, after obtaining multiple transformed images, the pixel coordinate transformation relationship between the image to be processed and each transformed image is saved. Using the pixel coordinate transformation relationship, the coordinates in the transformed images can be mapped to the image to be processed. It is understood that the multiple transformed images are all obtained by converting the size of the image to be processed. Each image in the image set (including the image to be processed and the multiple transformed images) has corresponding image features. Therefore, when mapping the second matching points in different transformed images to the image to be processed, there will be overlap between the mapped matching points in the image to be processed, and there will also be overlap between the mapped matching points in the image to be processed and the second matching points. Furthermore, considering the differences in image size between different images in the image set, there will be non-overlapping points between the mapped matching points in the image to be processed, and there will also be non-overlapping points between the mapped matching points in the image to be processed and the second matching points. These non-overlapping points all correspond to the image to be segmented in the template image, so these non-overlapping points can be used as pixel points for obtaining identification information, ensuring accurate identification information is obtained. Therefore, the present disclosure determines the union of the second matching point and the mapped matching point in the image to be processed as the first matching point, thereby ensuring that a sufficient number of first matching points are obtained and accurate identification information is obtained.

[0072] It can be understood that, through the above-mentioned scale conversion-based matching point (first matching point) extraction method, key point information of a specific target can be obtained. However, if the first matching point directly obtained is sent to the subsequent image segmentation model as identification information (prompts) for segmentation, on the one hand, it is easy to introduce edge information, which interferes with the segmentation of the target. On the other hand, more key points will make the target segmentation more detailed, resulting in further segmentation of the specific target. For example, in an image containing a desktop and a sign on the desktop, under the premise of only wanting to obtain the segmentation result of the desktop sign, after inputting more key points, it is easy to separate the product, text and background in the sign, rather than segmenting the sign as a whole, which is contrary to the original segmentation goal. Therefore, when designing the identification information, the present disclosure further processes the first matching point and designs a clustering algorithm to obtain more representative prompt information. The following embodiments of the present disclosure illustrate the method for determining identification information.

[0073] Figure 6 FIG. 1 is a flow chart showing a method for determining identification information based on a first matching point according to an exemplary embodiment. Figure 6 As shown, the method includes steps S401 to S402.

[0074] In step S401 , the first matching points are clustered by density clustering according to a preset number and a preset distance to obtain a plurality of first clusters consisting of the first matching points.

[0075] The number of first matching points in the first cluster is greater than or equal to a preset number, and the neighborhood radius of the first cluster is a preset distance.

[0076] In step S402 , identification information is determined according to the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed.

[0077] In the embodiment of the present disclosure, the first matching points are clustered by a density clustering algorithm (such as the DBSCAN density clustering algorithm), and the number of specific samples (i.e., first matching points) and the distance between the samples are set (such as setting the number of first matching points of each first cluster to be no less than 10, and the distance between the first matching points must be less than or equal to 150, where 150 is the Euclidean distance). The clustering algorithm is used to automatically find a suitable cluster, and the cluster with the most key point data in the first cluster (the second cluster) is selected as the location of the target to be segmented. Further, all the first clusters and the second clusters are combined to design identification information for image segmentation. The identification information in the present disclosure includes identification information in the form of points (identification points) and identification information in the form of boxes (identification boxes).

[0078] In the disclosed embodiment, the obtained identification points and identification boxes cover the location information of a specific category (target to be segmented) to a certain extent. By sending the identification points, identification boxes and the image to be processed into the image segmentation model, the segmentation results of the corresponding category can be output, thereby making up for the deficiency of the image segmentation model in the lack of specific semantic information and avoiding the inconvenience caused by manually setting the identification information.

[0079] In the embodiments of the present disclosure, identification information includes identification points and identification frames, and the identification points and identification frames need to be obtained in different ways. The following embodiments of the present disclosure illustrate the method of obtaining identification frames and identification points.

[0080] Figure 7 FIG. 1 is a flow chart showing a method for determining identification information according to an exemplary embodiment. Figure 7 As shown, the method includes steps S501 to S502.

[0081] In step S501, the first cluster containing the most first matching points among the multiple first clusters is determined as the second cluster, and the multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster are determined, and the center coordinates of the multiple pixel coordinates are determined as the identification point.

[0082] In step S502 , a minimum frame surrounding a plurality of first clusters is determined, and the minimum frame is determined as an identification frame.

[0083] In the embodiment of the present disclosure, the first matching point is a point in the image to be processed that has a matching relationship with the template image, and the density of the first matching points in the image to be processed can reflect the degree of feature matching between the image area where the first matching points are located and the template image. Therefore, the number of first matching points contained in the first cluster can reflect the degree of feature matching between the image area corresponding to the first cluster, and the image area corresponding to the first cluster containing the most first matching points has the highest degree of feature matching with the template image, and can best reflect the image area where the target to be segmented in the image to be processed is located. Therefore, the present disclosure determines the first cluster containing the most first matching points as the second cluster, and determines the identification point for identifying the target to be segmented based on the multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster.

[0084] In the disclosed embodiment, the identification point is obtained by calculating the center coordinates of the second cluster using the centroid method and using these center coordinates as the identification point. The identification box is obtained by finding the minimum circumscribed matrix that encloses the first cluster, that is, the minimum box that can enclose all samples in the cluster. The obtained minimum box is used as the identification box, and the pixel coordinates corresponding to the identification box are used to identify the target to be segmented.

[0085] In the disclosed embodiment, the acquired identification points and identification boxes cover the location information of a specific category (target to be segmented) to a certain extent. By acquiring multiple types of identification information, the accuracy of the image segmentation process is ensured.

[0086] In the embodiment of the present disclosure, after obtaining the prompt information including the identification point and the identification box, the prompt information can be input into the image segmentation model together with the image to be processed, and the image segmentation result output by the image segmentation model can be obtained.

[0087] Figure 8 FIG. 1 is a flow chart showing a method for segmenting an image to be processed according to identification information according to an exemplary embodiment. Figure 8 As shown, the method includes steps S601 to S602.

[0088] In step S601, an image to be processed is obtained, and identification information is obtained, where the identification information includes identification points and identification frames.

[0089] In step S602, the image to be processed, the identification points and the identification boxes are input into the image segmentation model, and the image area identified by the identification points and the identification boxes in the image to be processed is segmented.

[0090] Among them, the image segmentation model is used to segment the image to be processed according to the input identification points and identification boxes.

[0091] In the embodiment of the present disclosure, after obtaining the prompt information including the identification point and the identification box, the prompt information and the image to be processed are input into the image segmentation model. The image segmentation model performs image segmentation on the image area identified by the identification point and the identification box, obtains the image mask corresponding to the target to be segmented, and outputs the image to be processed including the image mask, thus completing the image segmentation.

[0092] In one implementation of the disclosed embodiment, the image segmentation model is a SAM (Severity Aspect Segmentation Model).

[0093] In the embodiment of the present disclosure, the SAM Segmentation Model is used as the image segmentation model of the image segmentation method involved in the present disclosure. The SAM model is an unsupervised image segmentation method that can segment any object in an image without any annotation. However, the SAM model lacks certain semantic information and cannot know the category information of the segmentation object: that is, since the SAM model performs panoramic segmentation on the image, it will segment each type of object contained in the image. However, if you want to obtain the segmentation result of a specific category, the user needs to manually input the corresponding prompt (point, box or text prompt word) of the category to obtain the segmentation result corresponding to the specific category in a targeted manner. The present disclosure obtains the identification information of a certain image to be processed through a template image, which makes up for the shortcomings of the SAM model. Based on the template image containing the target image to be segmented, the identification information used to segment the target to be segmented in the image to be processed is determined, so as to avoid the user setting the identification information for the target to be segmented by himself, avoid the inconvenience caused by manually setting the identification information, and improve the efficiency of image segmentation.

[0094] In an exemplary embodiment of the present disclosure, Figure 9 As shown in the flowchart of the image segmentation method in [1], image segmentation is performed as follows: a test image (the image to be processed) and a corresponding template image are obtained. The test image is multi-scaled to obtain a transformed image containing the original image and different scales (840, 1024, and 1280). The template image and the transformed images of different scales are input into a feature matching model (LightGlue). Based on the feature matching relationship, a set of matching points is obtained. These matching points are then input into a clustering module to obtain prompt information (center, box), which corresponds to the box and the five-pointed star in the image, respectively. The prompt information and the image to be processed are input into the segmentation module to obtain a segmentation mask corresponding to the area to be segmented, completing the image segmentation.

[0095] In an exemplary embodiment of the present disclosure, the processing results of the image segmentation method of the present disclosure are compared with the processing results of the image segmentation method in the related art. Figure 10 Schematic diagram of image segmentation processing results and Figure 11 The schematic diagram of the image segmentation processing result is shown in FIG. Figure 10 This is the processing result of the image segmentation method in the related technology. The points and boxes are prompt information. It can be seen that some key points fall on the edge area of ​​the desktop stand. After these key points are sent to the image segmentation model as prompt information, the edge area will be introduced into the segmentation result, making it impossible to accurately obtain the segmentation result of the desktop stand. Figure 11 This is the processing result of the image segmentation method in the present disclosure. The output of the clustering module (five-pointed star) is used as prompt information and fed into the image segmentation model, so the desktop stand can be accurately segmented, which illustrates the effectiveness of the clustering module.

[0096] In an exemplary embodiment of the present disclosure, the processing results of the image segmentation method of the present disclosure are described. Figure 12A 、 Figure 12B 、 Figure 12C and Figure 12D The processing results of segmenting the image to be segmented are shown in the schematic diagram. Figure 12A and 12B As shown in the figure, in the close-up scene, the image segmentation method in this disclosure can completely segment the desktop signboard, and the edge segmentation is relatively accurate. Figure 12C and Figure 12D As shown in the long-range image scene, the image segmentation method in the present disclosure can also accurately locate the position area of ​​the desktop stand when shooting from the side and segment it out, indicating that the algorithm has a certain robustness under different shooting angles.

[0097] In an embodiment of the present disclosure, in order to address the problem that related image segmentation models lack semantic information of specific categories, a large image segmentation model based on template image matching is proposed. The identification information of a specific category is automatically found by matching the features of the template image with the test image, and the identification information and the image to be processed are input into the image segmentation model to achieve efficient image segmentation. The entire process only requires a given template image to segment the objects of the template image in the image to be processed, so that the image segmentation model can be applied to segmentation in zero-sample situations and has a wide range of applicability. The present disclosure reduces the visual impact of the image caused by the shooting angle and distance by changing the size of the image to be processed, and then obtains matching points. It enriches the semantic information of the image through image changes of different scales, thereby outputting more representative feature matching points containing semantic information of specific categories. The present disclosure realizes the adaptive identification information design of each image through a clustering scheme, and inputs the identification information into the image segmentation model in the form of points and boxes to obtain accurate segmentation results of specific categories.

[0098] Figure 13 FIG. 1 is a block diagram of an image segmentation apparatus 100 according to an exemplary embodiment. Figure 13 The device includes a determining unit 101, a processing unit 102 and a segmenting unit 103.

[0099] The determining unit 101 is used to obtain an image to be processed and determine an object to be segmented in the image to be processed, and to determine a template image according to the object to be segmented. The template image is a preset image and includes the object image to be segmented.

[0100] The processing unit 102 is used to determine a first matching point in the image to be processed based on the image features of the target image to be segmented. The first matching point is a pixel point in the image to be processed corresponding to the target image to be segmented. Based on the first matching point, identification information is determined. The identification information is used to identify the image area corresponding to the target to be segmented in the image to be processed.

[0101] The segmentation unit 103 is configured to segment the image to be processed according to the identification information to obtain an object to be segmented.

[0102] In one embodiment, the processing unit 102 determines a first matching point in the image to be processed based on the image features of the target image to be segmented by performing image scale conversion on the image to be processed according to a plurality of preset image sizes, thereby obtaining a plurality of converted images corresponding to the plurality of preset image sizes. Based on the image features of the target image to be segmented, a second matching point corresponding to the target image to be segmented is determined in each image included in an image set, where the image set includes the target image to be segmented and the converted images, and the second matching point is a pixel point within the image corresponding to the target image to be segmented in each image included in the image set. Based on the second matching point, a first matching point is determined.

[0103] In one embodiment, the processing unit 102 is further configured to, after obtaining multiple transformed images, store the pixel coordinate transformation relationship between the image to be processed and each transformed image. The processing unit 102 determines the first matching point based on the second matching point in the following manner: based on the pixel coordinate transformation relationship between each transformed image in the multiple transformed images and the image to be processed, the second matching point in each transformed image is mapped to the image to be processed, and the second matching point mapped to the image to be processed is determined as the mapped matching point. The union of the second matching point in the image to be processed and the mapped matching point is determined as the first matching point.

[0104] In one embodiment, the processing unit 102 determines identification information based on the first matching points by clustering the first matching points using a density clustering method based on a preset number and a preset distance to obtain a plurality of first clusters composed of the first matching points, wherein the number of first matching points in the first clusters is greater than or equal to the preset number, and the neighborhood radius of the first clusters is a preset distance. The identification information is determined based on the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed.

[0105] In one embodiment, the identification information includes an identification point corresponding to the target to be segmented and an identification frame including the target to be segmented. The processing unit 102 determines the identification information in the following manner based on the distribution of the first matching points in the multiple first clusters and the distribution of the multiple first clusters in the image to be processed: the first cluster containing the most first matching points among the multiple first clusters is determined as the second cluster, and the multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster are determined, and the center coordinates of the multiple pixel coordinates are determined as the identification point. The minimum frame surrounding the multiple first clusters is determined, and the minimum frame is determined as the identification frame.

[0106] In one embodiment, the segmentation unit 103 segments the image to be processed based on the identification information to obtain the target to be segmented: the image to be processed, the identification points, and the identification box are input into an image segmentation model, and the image region identified by the identification points and the identification box in the image to be processed is segmented. The image segmentation model is used to segment the image to be processed based on the input identification points and identification box.

[0107] In one embodiment, the image segmentation model is a SAM (Everything Segmentation Model).

[0108] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0109] Figure 14 FIG2 is a block diagram of an apparatus 200 for image segmentation according to an exemplary embodiment. The apparatus 200 may be provided as a terminal. For example, the apparatus 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0110] Reference Figure 14 , apparatus 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0111] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0112] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0113] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 200.

[0114] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0115] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC) that is configured to receive external audio signals when the device 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 further includes a speaker for outputting audio signals.

[0116] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0117] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200. The sensor assembly 214 can also detect changes in the position of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200, and temperature changes of the device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 214 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0118] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0119] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0120] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as the memory 204 including instructions, which can be executed by the processor 220 of the apparatus 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0121] It is understood that in this disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of related objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0122] It will be further understood that the terms "first," "second," and the like are used to describe various types of information, but such information should not be limited to these terms. These terms are used solely to distinguish information of the same type from one another and do not indicate a particular order or level of importance. In fact, the terms "first," "second," and the like are fully interchangeable. For example, first information could be referred to as second information, and similarly, second information could be referred to as first information without departing from the scope of this disclosure.

[0123] It can be further understood that the terms "center", "longitudinal", "lateral", "front", "back", "up", "down", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation.

[0124] It is further understood that, unless otherwise specified, “connection” includes a direct connection where there are no other components between the two elements, and also includes an indirect connection where there are other elements between the two elements.

[0125] It is further understood that although operations are described in a particular order in the drawings in the embodiments of the present disclosure, this should not be construed as requiring that the operations be performed in the particular order shown or in a serial order, or that all of the operations shown be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0126] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0127] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. An image segmentation method, characterized in that: include: Acquire an image to be processed, and determine an object to be segmented in the image to be processed; Determining a template image according to the target to be segmented, wherein the template image is a preset image and includes the target image to be segmented; Determining a first matching point in the image to be processed according to image features of the target image to be segmented, where the first matching point is a pixel point in the image to be processed corresponding to the target image to be segmented; determining identification information according to the first matching point, where the identification information is used to identify an image area corresponding to the object to be segmented in the image to be processed; The object to be segmented is obtained by segmenting the image to be processed according to the identification information.

2. The method according to claim 1, characterized in that The step of determining a first matching point in the image to be processed according to the image features of the target image to be segmented includes: performing image scale conversion on the image to be processed according to a plurality of preset image sizes, respectively, to obtain a plurality of converted images corresponding to the plurality of preset image sizes; Determining, based on image features of the target image to be segmented, a second matching point corresponding to the target image to be segmented in each image included in an image set, wherein the image set includes the image to be processed and the plurality of converted images, and the second matching point is a pixel point in the image corresponding to the target image to be segmented in each image included in the image set; The first matching point is determined according to the second matching point.

3. The method according to claim 2, characterized in that The method further comprises: After obtaining the plurality of converted images, respectively saving the pixel coordinate conversion relationship between the image to be processed and each converted image; The determining the first matching point according to the second matching point includes: mapping a second matching point in each converted image to the image to be processed according to a pixel coordinate conversion relationship between each converted image in the plurality of converted images and the image to be processed, and determining the second matching point mapped to the image to be processed as a mapped matching point; The union of the second matching point in the image to be processed and the mapped matching point is determined as the first matching point.

4. The method according to claim 1, wherein The determining identification information according to the first matching point includes: Clustering the first matching points by density clustering according to a preset number and a preset distance to obtain a plurality of first clusters consisting of the first matching points, wherein the number of first matching points in the first clusters is greater than or equal to the preset number, and a neighborhood radius of the first cluster is the preset distance; The identification information is determined according to the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed.

5. The method according to claim 4, characterized in that The identification information includes an identification point corresponding to the object to be segmented and an identification frame including the object to be segmented; The determining the identification information according to the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed includes: Determine a first cluster containing the most first matching points among the multiple first clusters as a second cluster, determine multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster, and determine the center coordinates of the multiple pixel coordinates as the identification point; A minimum frame surrounding the plurality of first clusters is determined, and the minimum frame is determined as the identification frame.

6. The method according to claim 5, characterized in that Segmenting the target to be segmented in the image to be processed according to the identification information includes: Inputting the image to be processed, the identification points and the identification box into an image segmentation model, and segmenting the image area identified by the identification points and the identification box in the image to be processed; The image segmentation model is used to segment the image to be processed according to the input identification points and identification boxes.

7. The method according to claim 6, characterized in that The image segmentation model is the Everything Segmentation Model (SAM).

8. An image segmentation device, characterized in that: include: a determination unit, configured to obtain an image to be processed, determine an object to be segmented in the image to be processed, and determine a template image based on the object to be segmented, wherein the template image is a preset image and includes the object image to be segmented; a processing unit, configured to determine a first matching point in the image to be processed based on image features of the target image to be segmented, the first matching point being a pixel point in the image to be processed corresponding to the target image to be segmented, and to determine identification information based on the first matching point, the identification information being used to identify an image region in the image to be processed corresponding to the target image to be segmented; A segmentation unit is used to segment the to-be-segmented object in the to-be-processed image according to the identification information.

9. The device according to claim 8, characterized in that The processing unit determines a first matching point in the image to be processed according to the image features of the target image to be segmented in the following manner: performing image scale conversion on the image to be processed according to a plurality of preset image sizes, respectively, to obtain a plurality of converted images corresponding to the plurality of preset image sizes; Determining, based on image features of the target image to be segmented, a second matching point corresponding to the target image to be segmented in each image included in an image set, wherein the image set includes the image to be processed and the plurality of converted images, and the second matching point is a pixel point in the image corresponding to the target image to be segmented in each image included in the image set; The first matching point is determined according to the second matching point.

10. The device according to claim 9, characterized in that The processing unit is further configured to: After obtaining the plurality of converted images, respectively saving the pixel coordinate conversion relationship between the image to be processed and each converted image; The processing unit determines the first matching point according to the second matching point in the following manner: mapping a second matching point in each converted image to the image to be processed according to a pixel coordinate conversion relationship between each converted image in the plurality of converted images and the image to be processed, and determining the second matching point mapped to the image to be processed as a mapped matching point; The union of the second matching point in the image to be processed and the mapped matching point is determined as the first matching point.

11. The device according to claim 8, characterized in that The processing unit determines identification information according to the first matching point in the following manner: Clustering the first matching points by density clustering according to a preset number and a preset distance to obtain a plurality of first clusters consisting of the first matching points, wherein the number of first matching points in the first clusters is greater than or equal to the preset number, and a neighborhood radius of the first cluster is the preset distance; The identification information is determined according to the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed.

12. The device according to claim 11, characterized in that The identification information includes an identification point corresponding to the object to be segmented and an identification frame including the object to be segmented; The processing unit determines the identification information according to the distribution of the first matching points in the plurality of first clusters and the distribution of the plurality of first clusters in the image to be processed in the following manner: Determine a first cluster containing the most first matching points among the multiple first clusters as a second cluster, determine multiple pixel coordinates corresponding to the multiple first matching points contained in the second cluster, and determine the center coordinates of the multiple pixel coordinates as the identification point; A minimum frame surrounding the plurality of first clusters is determined, and the minimum frame is determined as the identification frame.

13. The device according to claim 8, characterized in that The segmentation unit segments the target to be segmented in the image to be processed according to the identification information in the following manner: Inputting the image to be processed, the identification points and the identification box into an image segmentation model, and segmenting the image area identified by the identification points and the identification box in the image to be processed; The image segmentation model is used to segment the image to be processed according to the input identification points and identification boxes.

14. The device according to claim 13, characterized in that The image segmentation model is the Everything Segmentation Model (SAM).

15. An electronic device, characterized in that: include: processor: a memory for storing processor-executable instructions; The processor is configured to execute the image segmentation method according to any one of claims 1 to 7.

16. A storage medium, characterized in that The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor, the processor is enabled to execute the image segmentation method according to any one of claims 1 to 7.

Images