An image instance segmentation method, an electronic device, and a medium

Abstract

The application discloses an image instance segmentation method, an electronic device and a medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a target image; inputting the target image into a trained image recognition model to obtain the position coordinates of the center pixel points of each object in the first image; the first image is an image processed by the trained image recognition model; performing size restoration on the first image, and obtaining the position coordinates of the center pixel points of each object in the target image according to the position coordinates of the center pixel points of each object in the first image and a target mapping relationship; inputting the target image and the position coordinates of the center pixel points of each object in the target image into a trained image segmentation model to obtain a target segmentation result. The application performs image instance segmentation based on the position coordinates of the center pixel points of each object and the target image, and the obtained target segmentation result is more accurate and has higher efficiency than the pixel-by-pixel segmentation method in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image instance segmentation method, an electronic device and a medium. BACKGROUND

[0002] Now the instance segmentation algorithm generally adopts the way of pixel-by-pixel segmentation, and the segmentation efficiency is very low, and the segmentation result will inevitably cause obvious deformation, that is, the pixels in an instance will exist the case of segmentation success or failure. Therefore, an efficient and accurate image instance segmentation method is urgently needed. SUMMARY

[0003] The purpose of the present application is to provide an image instance segmentation method, an electronic device and a medium, which can improve the efficiency and accuracy of image instance segmentation.

[0004] To achieve the above purpose, the present application provides the following scheme:

[0005] The present application provides an image instance segmentation method, which comprises:

[0006] Obtaining a target image;

[0007] Inputting the target image into a trained image recognition model to obtain the position coordinates of the center pixel points of each object in a first image; the first image is an image processed by the trained image recognition model; the trained image recognition model is a model trained by taking a sample image as input and taking the position coordinates of the center pixel points of each sample object in a first sample image corresponding to the sample image as a label;

[0008] Performing size restoration on the first image to obtain a second image, and obtaining the position coordinates of the center pixel points of each object in the target image according to the position coordinates of the center pixel points of each object in the first image and a target mapping relationship; the target mapping relationship is a mapping relationship between the second image and the target image, and the pixels of the second image are one-to-one mapped with the pixels of the target image;

[0009] Inputting the target image and the position coordinates of the center pixel points of each object in the target image into a trained image segmentation model to obtain a target segmentation result; the trained image segmentation model is a model trained by taking a sample image and the position coordinates of the center pixel points of each sample object in the sample image as input and taking a sample segmentation result as a label.

[0010] Optionally, after obtaining the target segmentation result, the method further comprises:

[0011] annotating a class and a class confidence of each of the objects in the target segmentation result, wherein the class and the class confidence of the object are obtained by a specific method as follows:

[0012] inputting the target image into the trained image recognition model to obtain a class and a class confidence of each of the objects in the first image.

[0013] Optionally, the image recognition model comprises an encoder and a decoder; the encoder comprises a first feature extraction network, a second feature extraction network and a feature integration network.

[0014] The first feature extraction network is configured to extract different scale features of the target image to obtain a first feature, a second feature and a third feature.

[0015] The second feature extraction network is configured to:

[0016] perform convolution operations on the first feature, the second feature and the third feature respectively to obtain a first convolution feature, a second convolution feature and a third convolution feature.

[0017] perform down-sampling operations on the third convolution feature to obtain a first down-sampled feature.

[0018] perform addition operations on the first down-sampled feature and the second convolution feature to obtain a first added feature.

[0019] perform down-sampling operations on the first added feature to obtain a second down-sampled feature.

[0020] perform addition operations on the second down-sampled feature and the first convolution feature to obtain a second added feature.

[0021] perform convolution operations on the first added feature, the second added feature and the third convolution feature respectively to obtain a fourth convolution feature, a fifth convolution feature and a sixth convolution feature.

[0022] perform splicing operations on the fourth convolution feature, the fifth convolution feature and the sixth convolution feature to obtain a spliced feature.

[0023] The feature integration network is configured to:

[0024] perform convolution operations on the spliced feature to obtain a spliced convolution feature.

[0025] perform a plurality of first convolution operations on the spliced convolution feature, and perform dimension change operations on the features after the plurality of first convolution operations to obtain a first dimension feature.

[0026] performing at least one second convolution operation on the spliced convolution feature, and performing a dimension change operation on the feature after the at least one second convolution operation to obtain a second dimension feature;

[0027] performing tensor multiplication on the first dimension feature and the second dimension feature to obtain an integrated multiplication result;

[0028] The decoder is configured to decode the integrated multiplication result to obtain the position coordinates of the center pixel point of each object in the first image.

[0029] Optionally, the first feature extraction network is a ResNet50 network.

[0030] Optionally, the decoder comprises a first full connection layer, a second full connection layer and a third full connection layer.

[0031] The first full connection layer is configured to decode the integrated multiplication result to obtain the position coordinates of the center pixel point of each object in the first image.

[0032] The second full connection layer is configured to decode the integrated multiplication result to obtain the category of each object in the first image.

[0033] The third full connection layer is configured to decode the integrated multiplication result to obtain the category confidence of each object in the first image.

[0034] Optionally, before the target image is input into the trained image recognition model, the method further comprises:

[0035] modifying the size of the target image to a set size.

[0036] Optionally, the image segmentation model is a SAM model.

[0037] The application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to enable the processor to perform the steps of the image instance segmentation method.

[0038] The application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by a processor to perform the image instance segmentation method.

[0039] According to the specific embodiments of the present application, the following technical effects are disclosed: the present application provides an image instance segmentation method, an electronic device and a medium, the method comprising: obtaining a target image; inputting the target image into a trained image recognition model to obtain the position coordinates of the center pixel points of each object in the first image; the first image is an image processed by the trained image recognition model; performing size restoration on the first image to obtain a second image, and obtaining the position coordinates of the center pixel points of each object in the target image according to the position coordinates of the center pixel points of each object in the first image and a target mapping relationship; the target mapping relationship is the mapping relationship between the second image and the target image, and the pixels of the second image are one-to-one mapped with the pixels of the target image; inputting the target image and the position coordinates of the center pixel points of each object in the target image into a trained image segmentation model to obtain a target segmentation result. The present application obtains more semantic information (the position coordinates of the center pixel points of each object) in the target image through the image recognition model, performs image instance segmentation based on the position coordinates of the center pixel points of each object and the target image, and the obtained target segmentation result is more accurate and has higher efficiency than the pixel-by-pixel segmentation method of the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 The image instance segmentation method flowchart provided by the embodiments of the present application;

[0042] Figure 2 The image recognition model and image segmentation model structure diagram provided by the embodiments of the present application;

[0043] Figure 3 The ResNet50 network structure diagram provided by the embodiments of the present application;

[0044] Figure 4 The feature integration network structure diagram provided by the embodiments of the present application;

[0045] Figure 5 The decoder structure diagram provided by the embodiments of the present application;

[0046] Figure 6 The structure diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0047] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0048] For the current instance segmentation algorithm, the pixel-by-pixel segmentation method is generally used, which is low in segmentation efficiency, and there is a case that the segmentation of pixels within an instance succeeds or fails. The emergence of the SAM (Segment Anything model) model changes this situation. The SAM model can segment objects in an image and has almost no obvious deformation, but cannot obtain semantic information therein. With the advantage of light weight, the SAM model has great potential for instance segmentation.

[0049] On this basis, a light-weight instance extraction algorithm is developed to achieve the purpose of high-speed semantic segmentation. Existing instance segmentation algorithms generally consume a large amount of computing power and have a very long segmentation time, which cannot meet the actual use requirements. The present application is established on the basis of the SAM model, and is no longer a pixel-level network, but only needs to perceive the instance center coordinates and semantics, so it no longer needs a large amount of computing power support, and thus can greatly reduce the segmentation time, get rid of the dependence on large servers, and can run only by relying on a domain controller, which can meet the actual requirements.

[0050] The purpose of the present application is to provide an image instance segmentation method, an electronic device and a medium, which obtain more semantic information (position coordinates of the center pixel point of each object) in a target image through an image recognition model, perform image instance segmentation based on the position coordinates of the center pixel point of each object and the target image, and obtain a more accurate target segmentation result, and the method is more efficient than the pixel-by-pixel segmentation method of the prior art.

[0051] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] As shown in Figure 1 The present application provides an image instance segmentation method, which comprises:

[0053] S1: obtaining a target image.

[0054] S2: inputting the target image into the trained image recognition model to obtain the position coordinates of the center pixel points of each object in the first image; the first image is an image processed by the trained image recognition model; the trained image recognition model is a model trained by taking a sample image as input and taking the position coordinates of the center pixel points of each sample object in a first sample image corresponding to the sample image as a label.

[0055] S3: performing size restoration on the first image to obtain a second image, and obtaining the position coordinates of the center pixel points of each object in the target image according to the position coordinates of the center pixel points of each object in the first image and a target mapping relationship; the target mapping relationship is a mapping relationship between the second image and the target image, and the pixels of the second image are one-to-one mapped with the pixels of the target image.

[0056] S4: inputting the target image and the position coordinates of the center pixel points of each object in the target image into a trained image segmentation model to obtain a target segmentation result; the trained image segmentation model is a model trained by taking a sample image and the position coordinates of the center pixel points of each sample object in the sample image as input and taking a sample segmentation result as a label.

[0057] Before the target image is input into the trained image recognition model, the method further comprises:

[0058] The size of the target image is modified to a set size.

[0059] Specifically, in order to ensure that the image is not distorted, a padding method is used when the image size is changed, that is, in the case of fixed length or height, the center of the image is fixed, and white pixels are added in the height or length direction to meet the image size requirement. The size of the changed image is 224x224. 224x224 is not the only limit of the target image size, and the changed image size can be set by the user.

[0060] The image recognition model includes an encoder and a decoder; as Figure 2 As shown in the figure, the encoder includes a first feature extraction network, a second feature extraction network, and a feature integration network.

[0061] The first feature extraction network is configured to extract different scale features of the target image to obtain a first feature, a second feature, and a third feature.

[0062] In this embodiment, the first feature extraction network is a ResNet50 network, and the image features are preliminarily extracted through the ResNet50 network. The structure of the ResNet50 network is as shown in Figure 3As shown, the first feature extraction network includes a convolutional layer, a maximum pooling layer, three first extraction modules, four second extraction modules, six third extraction modules, and three fourth extraction modules connected in sequence.

[0063] The ResNet50 network first performs a convolution operation on an input target image (224x224x3 in size), where the convolution kernel of the convolutional layer is 7x7, and the step is 2, so that the image size is changed to 112x112x64.

[0064] Then, the image size is further compressed by a maximum pooling layer with a step of 2, so that a feature layer with a size of 56x56x64 is obtained, denoted as C1.

[0065] In the next three first extraction modules, two branches are involved. In the branch with two convolution modules, there are three convolution operations with convolution kernel sizes of 1x1, 3x3, and 1x1, respectively. The convolution kernel size of the other branch is 1x1, and the step is 1. After the three extraction modules, the feature size is changed to 56x56x256, denoted as C2.

[0066] After that, four second extraction modules are processed, which are similar to the method of obtaining C2, except that the convolution step in the first extraction module in the branch with two convolution modules is 2, and the convolution step in the other branch is also 2, so as to change the size of the feature layer. After processing, the feature size is changed to 28x28x512, denoted as C3 (i.e., the third feature).

[0067] The subsequent processing method is similar to the method of obtaining C3, except that the number of extraction modules is different. After C3 is processed by six third extraction modules, the size is changed to 14x14x1024, denoted as C4 (i.e., the second feature).

[0068] After C4 is processed by three fourth extraction modules, the size is changed to 7x7x2048, denoted as C5 (i.e., the first feature).

[0069] The second feature extraction network is configured to:

[0070] The first feature, the second feature, and the third feature are respectively subjected to a convolution operation to obtain a first convolution feature, a second convolution feature, and a third convolution feature.

[0071] The third convolution feature is subjected to a down-sampling operation to obtain a first down-sampling feature.

[0072] The first down-sampling feature and the second convolution feature are subjected to an addition operation to obtain a first addition feature.

[0073] The first addition feature is subjected to a down-sampling operation to obtain a second down-sampling feature.

[0074] performing an addition operation on the second down-sampling feature and the first convolution feature to obtain a second addition feature.

[0075] performing a convolution operation on the first addition feature, the second addition feature and the third convolution feature respectively to obtain a fourth convolution feature, a fifth convolution feature and a sixth convolution feature.

[0076] performing a splicing operation on the fourth convolution feature, the fifth convolution feature and the sixth convolution feature to obtain a splicing feature.

[0077] The structure of the second feature extraction network is as shown in Figure 2 The method focuses on feature extraction, rather than pixel-by-pixel resolution, and therefore, down-sampling is used to replace up-sampling operation, and the features of C3 and C4 are gradually transmitted to the C5 layer.

[0078] The C3, C4 and C5 feature layers are subjected to convolution operation (convolution kernel size is 1x1, and step is 1), and the number of channels is unified, and the sizes are changed to 7x7x512, 14x14x512 and 28x28x512 respectively, which are denoted as a third convolution feature C31, a second convolution feature C41 and a first convolution feature C51 respectively. The third convolution feature C31 is subjected to down-sampling operation and becomes 14x14x512, and a first down-sampling feature is obtained. The first down-sampling feature is added to the C41, and the size is still 14x14x512, which is denoted as a first addition feature C42. The first addition feature C42 is subjected to down-sampling operation and becomes 7x7x512, and a second down-sampling feature is obtained. The second down-sampling feature is added to the C51, and the size is still 7x7x512, which is denoted as a second addition feature C52.

[0079] The third convolution feature C31 is subjected to two convolution operations with a convolution kernel size of 3x3 and a step of 2, and the size is changed to 7x7x512, which is denoted as a sixth convolution feature C32. The first addition feature C42 is subjected to a convolution operation with a convolution kernel size of 3x3 and a step of 2, and the size is changed to 7x7x512, which is denoted as a fourth convolution feature C43. The second addition feature C52 is subjected to a convolution operation with a convolution kernel size of 1x1 and a step of 1, and the size remains unchanged, which is denoted as a fifth convolution feature C53. After splicing processing of C32, C43 and C53, a feature layer with a size of 7x7x1536 is obtained, which is denoted as a splicing feature.

[0080] The structure of the feature integration network is as shown in Figure 4 The feature integration network is used for:

[0081] performing a convolution operation on the splicing feature to obtain a splicing convolution feature.

[0082] The spliced convolutional features are subjected to a plurality of first convolutional operations, and the features after the plurality of first convolutional operations are subjected to a dimension change operation to obtain first dimension features.

[0083] The spliced convolutional features are subjected to at least one second convolutional operation, and the features after the at least one second convolutional operation are subjected to a dimension change operation to obtain second dimension features.

[0084] The first dimension features and the second dimension features are subjected to tensor multiplication to obtain an integrated multiplication result.

[0085] After the splicing processing, the features of the image are preliminarily extracted. In order to make the extracted features have practical significance, an integration operation is performed.

[0086] First, the feature layer of 7x7x1536 (i.e., the spliced features) is subjected to a convolutional processing with a convolution kernel of 1x1 and a step of 1 to change the number of channels, and a feature layer of 7x7x256 (i.e., the spliced convolutional features) is obtained.

[0087] Then, in one branch, after being subjected to processing of four integration modules (in the first convolutional operation in which, the convolution kernel is 1x1 and the step is 1), the size of the feature layer becomes 7x7x100, and after a dimension change, the processing is performed to a size of 100x49 to obtain the first dimension features.

[0088] In another branch, after being subjected to a second convolutional operation with a convolution kernel of 1x1 and a step of 1, the size of the feature becomes 7x7x258, and after a Sigmoid activation and a dimension change, a feature layer of 49x256 is obtained to obtain the second dimension features.

[0089] The results of the two branches (i.e., the first dimension features and the second dimension features) are subjected to tensor multiplication to obtain an integrated result (the integrated multiplication result) of a final size of 100x256, wherein 100 represents that the upper limit of the instances after segmentation is 100, i.e., the number of feature layers, and 256 is the number of channels of each feature layer, which can be regarded as the result of the encoder.

[0090] The decoder is configured to decode the integrated multiplication result to obtain the position coordinates of the center pixel points of each object in the first image.

[0091] In one specific example, after obtaining the target segmentation result, further comprising:

[0092] The class and class confidence of each object in the target segmentation result are labeled, wherein the specific method for obtaining the class and class confidence of the object is:

[0093] Input the target image into the trained image recognition model to obtain the category and category confidence of each object in the first image.

[0094] As shown in the formula (1), the decoder includes a first fully connected layer, a second fully connected layer, and a third fully connected layer. Figure 5

[0095] The first fully connected layer is configured to decode the integrated multiplication result to obtain the position coordinates of the center pixel point of each object in the first image; the second fully connected layer is configured to decode the integrated multiplication result to obtain the category of each object in the first image; and the third fully connected layer is configured to decode the integrated multiplication result to obtain the category confidence of each object in the first image.

[0096] The result after the feature integration basically has a preliminary practical meaning, but still needs to be further decoded to obtain a specific result.

[0097] After passing through the three fully connected layers respectively, the decoding results with sizes of 100x784, 100x80, and 100x1 can be obtained. Among them, 100x784 represents the position of the center pixel point of the object in the second image extracted by the 100 feature layers, 100x80 represents the object category number extracted by the 100 feature layers, and 100x1 represents the category confidence of the 100 feature layers.

[0098] After obtaining the position coordinates of the center pixel point of the object in the second image, the size of the second image is changed to 28x28, and is further expanded to 224x224 by using the up-sampling method. The position coordinates of the point on the target image are obtained by size reduction, that is, the image with the changed size of 224x224 (the second image) is mapped to the input target image, and the segmented image mask is obtained by processing the image by using the image segmentation model, and the segmentation result is further completed. In this embodiment, the image segmentation model is a SAM model. The SAM model is an open source model which can automatically segment the object represented by the center pixel point by only relying on the input center point data, and outputs the target segmentation result by using the deep learning method.

[0099] After obtaining the target segmentation result, the category information and the confidence are converted into text, and the program is used to draw the text at the center point position of the image mask segmentation result, and finally the instance segmentation result with the segmentation result, semantic information, and confidence is formed, and the image drawing is completed.

[0100] In a specific embodiment, the image instance segmentation method in the present application can be applied to various image instance segmentations, such as human-vehicle instance segmentation, scene segmentation, and instance segmentation of dense crowds.

[0101] ​The image segmentation can be used in various fields, and in the automatic driving field, each vehicle and pedestrian can be segmented out, so that the trajectories of the vehicles and pedestrians are predicted subsequently, and the danger is avoided.

[0102] The application relies on a SegmentAnything model, and proposes a semantic core extraction method. ResNet-50 and an improved reverse feature pyramid model are used as an encoder, unlike traditional methods, which do not rely on the downward extraction of a traditional feature pyramid model, gradually expand the perception range, but use the reverse feature pyramid model to transmit the low-layer pixel-by-pixel feature extraction result upward, not to realize pixel-level perception, but to achieve instance perception, realize coarse extraction of instance information, and further realize pixel-by-pixel instance segmentation by using the SegmentAnything model.

[0103] The application has the following beneficial effects:

[0104] (1) The model is relatively lightweight: the upsampling of the feature pyramid network structure is converted into downsampling, a large number of parameters required in the network are cut down, and since the entire network takes the extraction of the center point as the primary purpose, and is not pixel-by-pixel classification, the number of channels in the entire network is greatly reduced, the network is more lightweight, and does not rely on a large server, but only relies on a domain controller to run.

[0105] (2) The segmentation result is relatively better: after the center point is extracted, the SAM model is introduced for processing, the SAM model has good segmentation result, and compared with pixel-by-pixel segmentation, the segmentation result has obvious deformation, the SAM model has good visual sense, and does not have obvious deformation problems.

[0106] (3) The non-maximum suppression algorithm is introduced in instance segmentation, so that the uniqueness of the instance center point is ensured.

[0107] The application further provides an electronic device including a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the image instance segmentation method.

[0108] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of an electronic device provided by the application. As Figure 6As shown, the electronic device 1000 can include a processor 1001, a network interface 1004 and a memory 1005, in addition, the electronic device 1000 can further include a user interface 1003, and at least one communication bus 1002. Wherein the communication bus 1002 is used to realize the connection communication between the components. Wherein the user interface 1003 can include a display, a keyboard, and the optional user interface 1003 can further include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a non-volatile memory, for example, at least one disk storage. The memory 1005 can optionally be at least one storage device located away from the aforementioned processor 1001. For example Figure 6 As shown, the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module and a device control application.

[0109] In Figure 6 As shown in the electronic device 1000, the network interface 1004 can provide network communication function; while the user interface 1003 is mainly used to provide the interface for the user to input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to realize the image instance segmentation method described in the above embodiment, which will not be repeated here.

[0110] The application further provides a readable storage medium, the readable storage medium stores a computer program, the computer program is suitable for being loaded by a processor and executing the image instance segmentation method described in the above embodiment, which will not be repeated here.

[0111] The above program can be deployed on an electronic device for execution, or deployed on multiple electronic devices located in one place for execution, or on multiple electronic devices distributed in multiple places and interconnected through a communication network for execution, and the multiple electronic devices distributed in multiple places and interconnected through a communication network can constitute a blockchain network.

[0112] The readable storage medium can be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The readable storage medium is used to store the computer program and other programs and data required by the electronic device. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0113] The various embodiments are described in the specification by way of progression, each building on the last to facilitate ease of understanding. The same reference numerals are used throughout the drawings and like structure are denoted with like reference numerals.

[0114] The principles and implementations of the present application are described in the specification by way of specific examples, and the above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. An image instance segmentation method, characterized in that, The method comprises: acquiring a target image; inputting the target image into a trained image recognition model to obtain the position coordinates of the center pixel points of each object in a first image; the first image is an image processed by the trained image recognition model; the trained image recognition model is a model trained by taking a sample image as input and taking the position coordinates of the center pixel points of each sample object in a first sample image corresponding to the sample image as a label; the image recognition model comprises an encoder and a decoder; the encoder comprises a first feature extraction network, a second feature extraction network and a feature integration network; wherein the first feature extraction network is configured to extract different scale features of the target image to obtain a first feature, a second feature and a third feature; the second feature extraction network is configured to: perform convolution operations on the first feature, the second feature and the third feature respectively to obtain a first convolution feature, a second convolution feature and a third convolution feature; perform down-sampling operations on the third convolution feature to obtain a first down-sampled feature; perform addition operations on the first down-sampled feature and the second convolution feature to obtain a first added feature; perform down-sampling operations on the first added feature to obtain a second down-sampled feature; perform addition operations on the second down-sampled feature and the first convolution feature to obtain a second added feature; perform convolution operations on the first added feature, the second added feature and the third convolution feature respectively to obtain a fourth convolution feature, a fifth convolution feature and a sixth convolution feature; perform splicing operations on the fourth convolution feature, the fifth convolution feature and the sixth convolution feature to obtain a spliced feature; the feature integration network is configured to: perform convolution operations on the spliced feature to obtain a spliced convolution feature; perform a plurality of first convolution operations on the spliced convolution feature, and perform dimension change operations on the features after the plurality of first convolution operations to obtain a first dimension feature; perform at least one second convolution operation on the spliced convolution feature, and perform dimension change operations on the features after the at least one second convolution operation to obtain a second dimension feature; perform tensor multiplication on the first dimension feature and the second dimension feature to obtain an integrated multiplication result; the decoder is configured to decode the integrated multiplication result to obtain the position coordinates of the center pixel points of each object in the first image; perform size restoration on the first image to obtain a second image, and obtain the position coordinates of the center pixel points of each object in the target image according to the position coordinates of the center pixel points of each object in the first image and a target mapping relationship; the target mapping relationship is a mapping relationship between the second image and the target image, and the pixels of the second image are one-to-one mapped with the pixels of the target image; inputting the target image and position coordinates of the center pixel of each object in the target image into a trained image segmentation model to obtain a target segmentation result; the trained image segmentation model is a model trained by taking a sample image and position coordinates of the center pixel of each sample object in the sample image as input and taking a sample segmentation result as a label; and the image segmentation model is a SAM model.

2. The image instance segmentation method of claim 1, wherein, After obtaining the target segmentation result, the method further includes: annotating the category and category confidence of each object in the target segmentation result, wherein the category and category confidence of the object are obtained by using the following specific method: inputting the target image into a trained image recognition model to obtain the category and category confidence of each object in the first image.

3. The image instance segmentation method of claim 1, wherein, The first feature extraction network is a ResNet50 network.

4. The image instance segmentation method of claim 1, wherein, The decoder includes a first full connection layer, a second full connection layer, and a third full connection layer. The first full connection layer is configured to decode the integrated multiplication result to obtain the position coordinates of the center pixel of each object in the first image. The second full connection layer is configured to decode the integrated multiplication result to obtain the category of each object in the first image. The third full connection layer is configured to decode the integrated multiplication result to obtain the category confidence of each object in the first image.

5. The image instance segmentation method of claim 1, wherein, Before inputting the target image into the trained image recognition model, the method further includes: modifying the size of the target image to a set size.

6. An electronic device, comprising: The computer program is loaded and executed by the processor to make the processor execute the steps of the method in any one of claims 1-5.

7. A readable storage medium, characterized by, The readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor to execute the method in any one of claims 1-5.

Images