A celestial body crater image instance segmentation method and system based on a text prompt

Abstract

The application discloses a celestial body meteor crater image instance segmentation method and system based on a text prompt, relates to the technical field of image segmentation, and comprises the following steps: first, receiving a celestial body meteor crater image, using a SAM large model to perform image segmentation on the celestial body meteor crater image to obtain an image segmentation mask; filtering and screening the image segmentation mask, filtering out non-circular and non-elliptical image segmentation masks to obtain a filtered image segmentation mask; receiving a text prompt, using an image-text pre-training model to perform feature extraction on the filtered image segmentation mask and the text prompt to obtain M text feature vectors and image feature vectors, performing similarity calculation on the M text feature vectors and image feature vectors to obtain M similarity scores, selecting a text feature vector with the largest similarity score with the image feature vector as a classification result, and realizing instance segmentation on the celestial body meteor crater image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to a celestial body crater image instance segmentation method and system based on text prompts. BACKGROUND

[0002] There are a large number of craters on celestial bodies such as Mars and the moon. By studying the craters on celestial bodies, we can understand the geological features of the surface of the celestial body, which helps to reveal the formation and evolution of the celestial body. Accurate segmentation and detection of celestial body craters provide safety guarantees for deep space exploration missions such as accurate landing of spacecraft. In recent years, researchers have proposed most methods for image segmentation, from convolutional neural networks combined with traditional edge detection algorithms to hybrid supervised and unsupervised clustering algorithms, and the accuracy of image segmentation has been rapidly improved. However, the current methods still have defects and deficiencies, and most existing model algorithms rely on a large amount of data labeling. Although some models perform well after training on a certain celestial body data, they perform poorly on other celestial body data sets, i.e., lack the ability of zero-shot segmentation. Recently, META AI has opened source a basic model SAM for computer vision and image segmentation, which has trained more than 1 billion masks and can perform instant zero-shot image segmentation. Although SAM is good at spatial understanding, it has limitations in semantic understanding. Some existing multi-modal large models, such as CLIP, ALBEF, BLIP, etc., provide powerful image backbone for semantic understanding, but cannot directly perform segmentation tasks. SUMMARY

[0003] To solve the deficiencies mentioned in the background art, the purpose of the present application is to provide a celestial body crater image instance segmentation method and system based on text prompts, which can realize accurate instance segmentation of celestial body crater images and provide technical support for deep space exploration missions.

[0004] In the first aspect, the purpose of the present application can be achieved by the following technical solution: a celestial body crater image instance segmentation method based on text prompts, the method comprising the following steps:

[0005] Receiving a celestial body crater image, performing image segmentation on the celestial body crater image to obtain an image segmentation mask;

[0006] Filtering and screening the image segmentation mask to filter out non-circular and non-elliptical image segmentation masks to obtain a filtered image segmentation mask;

[0007] The filtered image segmentation mask is subjected to feature extraction with the text prompt to obtain M text feature vectors and image feature vectors, similarity calculation is performed on the M text feature vectors and image feature vectors to obtain M similarity scores, a text feature vector with the largest similarity score with the image feature vector is selected as a classification result, and instance segmentation of the celestial body crater image is realized.

[0008] With reference to the first aspect, in some implementations of the first aspect, the method further includes that the celestial body crater image is obtained by being photographed by a detector camera.

[0009] With reference to the first aspect, in some implementations of the first aspect, the method further includes that the image segmentation of the celestial body crater image is performed by inputting the celestial body crater image into a pre-established SAM model.

[0010] With reference to the first aspect, in some implementations of the first aspect, the method further includes that the pre-established SAM model includes an image encoder, a prompt encoder and a mask decoder; and the prompt encoder adopts a text prompt-based input mode.

[0011] With reference to the first aspect, in some implementations of the first aspect, the method further includes that the celestial body crater image is subjected to the image encoder, the prompt encoder and the mask decoder respectively, and an image segmentation mask is output from the mask decoder.

[0012] With reference to the first aspect, in some implementations of the first aspect, the method further includes that the feature extraction of the text prompt is performed by using a multi-modal image-text large model of CLIP, ALBEF and BLIP.

[0013] With reference to the first aspect, in some implementations of the first aspect, the method further includes that the pre-established SAM model has an expression as follows:

[0014] F img =Φ i-enc (I)

[0015] T prompt =Φ p-enc ({p})

[0016] Masks=Φ m-dec (F img +F c-mask ,T prompt

[0017] I represents an input celestial body crater image, Φ i-enc represents an image encoder, F img ​represents the intermediate features of the input crater image obtained by the image encoder. p represents the text prompt, and p-enc represents the prompt encoder, T prompt represents the embedding tokens output by the prompt encoder after the text prompt passes through the prompt encoder, and m-dec represents the mask decoder, F c-mask represents the input mask, and Masks represents the result output by the mask decoder.

[0018] In combination with the first aspect, in some implementations of the first aspect, the method further comprises: the filtering and screening of the image segmentation mask adopts a roundness screening:

[0019] wherein the output of the SAM model contains the pixel area S of the image segmentation mask, and the radius r is obtained The circumference is represented as The image mask region output by the SAM is represented as p, when n=d / p approaches 1, the shape region is determined to be a circle; define θ=a / b, where a and b represent the long semi-axis and short semi-axis of the ellipse respectively, when n=θ=1, the shape is determined to be a circle. For the judgment of the elliptical shape, the area of the ellipse is represented as W=πab, and if the image segmentation mask shape is an ellipse, then q=W / S=1 is satisfied.

[0020] In combination with the first aspect, in some implementations of the first aspect, the method further comprises: the calculation process of the similarity calculation of the M text feature vectors and the image feature vector to obtain M similarity scores:

[0021] The similarity calculation is performed in the form of cosine similarity, and the probability mapping of classification comes from the Softmax processing of the similarity score, which is represented as:

[0022]

[0023] wherein score i represents the probability of the image segmentation mask corresponding to the i-th text, and there are j classes in total, and j texts.

[0024] Secondly, in order to achieve the above-mentioned purpose, the application discloses a celestial body crater image instance segmentation system based on a text prompt, comprising:

[0025] An image segmentation module is configured to receive a celestial body crater image, perform image segmentation on the celestial body crater image, and obtain an image segmentation mask.

[0026] A filtering and screening module is configured to filter and screen the image segmentation mask, filter out non-circular and non-elliptical image segmentation masks, and obtain a filtered image segmentation mask.

[0027] The extraction calculation module is configured to receive a text prompt, perform feature extraction on the filtered image segmentation mask and the text prompt to obtain M text feature vectors and an image feature vector, perform similarity calculation on the M text feature vectors and the image feature vector to obtain M similarity scores, select a text feature vector with the largest similarity score with the image feature vector as a classification result, and realize instance segmentation on the celestial body crater image.

[0028] In combination with the second aspect, in some implementations of the second aspect, the system further includes that the celestial body crater image is obtained by a detector camera;

[0029] Or the image segmentation module performs image segmentation on the celestial body crater image by inputting the celestial body crater image into a pre-established SAM model for image segmentation.

[0030] Or the pre-established SAM in the image segmentation module includes an image encoder, a prompt encoder, and a mask decoder; the prompt encoder adopts a text prompt-based input mode.

[0031] Or the expression of the pre-established SAM model in the image segmentation module is as follows:

[0032] F img =Φ i-enc (I)

[0033] T prompt =Φ p-enc ({p})

[0034] Masks=Φ m-dec (F img +F c-mask ,T prompt )

[0035] I represents an input celestial body crater image, Φ i-enc represents an image encoder, F img represents an intermediate feature obtained by inputting the crater image into the image encoder. p represents a text prompt, Φ p-enc represents a prompt encoder, T prompt represents an embedded token output by the text prompt after inputting the text prompt into the prompt encoder, Φ m-dec represents a mask decoder, F c-mask represents an input mask, and Masks represents a result output by the mask decoder.

[0036] Preferably, the celestial body crater image in the image segmentation module is respectively input into the image encoder, the prompt encoder, and the mask decoder, and the image segmentation mask is output from the mask decoder.

[0037] Preferably, the screening method of filtering the image segmentation mask in the mask screening module adopts roundness screening:

[0038] Wherein, the output of the SAM model contains the pixel area S of the image segmentation mask, and the radius R is obtained The circumference is expressed as The image mask area circumference output by the SAM is expressed as p, when n=d / p approaches 1, the shape area is determined to be a circle; define θ=a / b, where a and b represent the long semi-axis and the short semi-axis of the ellipse respectively, when n=θ=1, the shape is determined to be a circle; for the judgment of the elliptical shape, the area of the ellipse is expressed as W=πab, if the image segmentation mask shape is an ellipse, then q=W / S=1 is satisfied.

[0039] Preferably, the feature extraction of the text prompt in the extraction calculation module is performed by using the multi-modal image-text large model of CLIP, ALBEF and BLIP.

[0040] Preferably, the calculation process of obtaining M similarity scores by calculating the similarity between M text feature vectors and image feature vectors in the extraction calculation module is as follows:

[0041] The similarity is calculated by using the cosine similarity, and the probability mapping of classification comes from the Softmax processing of the similarity score, which is expressed as:

[0042]

[0043] Wherein, score i Indicates the probability of the image segmentation mask corresponding to the i-th text, and here there are j classes, that is, j texts.

[0044] The beneficial effects of the present application are as follows:

[0045] The present application fully combines the current image segmentation large model SAM and the multi-modal image-text large model CLIP, ALBEF and BLIP, and can efficiently perform instance segmentation of meteor crater images by combining the advantages of the two. The present application provides a screening processing method for image segmentation mask, which can further process the meteor crater image mask segmented by SAM to make it more consistent with the ideal segmentation effect. The present application can be used for instance segmentation of celestial meteor crater images, without the need to construct a large-scale training set or train the model, saving a lot of manpower and material resources. BRIEF DESCRIPTION OF DRAWINGS

[0046] The drawings needed in the embodiments or prior art description will be briefly introduced as follows, and obviously, other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 is a schematic diagram of the method flow of the present application;

[0048] Figure 2 is a schematic diagram of the workflow of the present application;

[0049] Figure 3 is a schematic diagram of the system structure of the present application;

[0050] Figure 4 is a schematic diagram of the mask screening module flow of the present application;

[0051] Figure 5 is a schematic diagram of a common meteorite crater shape provided by the present application;

[0052] Figure 6 is a schematic diagram of the instance segmentation effect of the present application DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0054] Embodiment one:

[0055] Next, the related terms involved in the embodiments of the present application are introduced:

[0056] Meteorite crater: Meteorite crater refers to a crater formed by a high-speed and violent impact of an extraterrestrial meteorite (star or meteor) on the ground, which is mostly circular, elliptical, or forms a radial and ring-shaped structure. The crater is also called a meteorite crater. The inner wall of the meteorite crater is steep and the outer edge is gentle. Large meteorite craters often have a prominent central peak, and there are often angular breccia and high metamorphic minerals in the crater.

[0057] Image segmentation is a technique and process of dividing an image into a number of specific regions with unique properties and proposing a target of interest. It is a key step from image processing to image analysis. Existing image segmentation methods mainly include the following categories: threshold-based segmentation method, region-based segmentation method, edge-based segmentation method, and segmentation method based on specific theory, etc. From a mathematical point of view, image segmentation is a process of dividing a digital image into mutually disjoint regions. The process of image segmentation is also a labeling process, that is, pixels belonging to the same region are assigned the same number.

[0058] Segmentation mask is a technique in computer vision that precisely separates objects in an image from the background. It classifies each pixel, achieving fine-grained division of image regions. Each pixel is assigned a label indicating whether it belongs to the foreground or background, or different object categories. Such label information forms a two-dimensional matrix, i.e., a segmentation mask.

[0059] The generation of segmentation mask relies on deep learning and artificial intelligence technology. A neural network is trained on a large amount of image data to learn and understand various object features. During the calculation process, the neural network gradually extracts image feature information through convolution, pooling, and other operations on the input image. Finally, the network output mask can accurately describe the location and boundary of different objects in the image.

[0060] As shown in Figure 1 , a text prompt-based celestial body crater image instance segmentation method is characterized in that the method comprises the following steps:

[0061] Receiving a celestial body crater image, performing image segmentation on the celestial body crater image, and obtaining an image segmentation mask:

[0062] The celestial body crater image is obtained by a detector camera;

[0063] Wherein, the image segmentation on the celestial body crater image is performed by inputting the celestial body crater image into a pre-established SAM model for image segmentation;

[0064] The pre-established SAM model includes an image encoder, a prompt encoder, and a mask decoder; the prompt encoder adopts a text prompt-based input method:

[0065] The expression of the pre-established SAM model is as follows:

[0066] F img =Φ i-enc (I)

[0067] T prompt =Φ p-enc ({p})

[0068] Masks=Φ m-dec (F img +F c-mask , T prompt )

[0069] I represents the input celestial body crater image, Φ i-enc represents the image encoder, F img represents the intermediate features obtained by inputting the crater image into the image encoder. p represents the text prompt, and Φp-enc denotes a prompt encoder, T prompt denotes the embedding tokens output by the prompt encoder, Φ m-dec denotes a mask decoder, F c-mask denotes an input mask, Masks denotes the result output by the mask decoder.

[0070] Further, the celestial body crater image is output from the mask decoder after passing through the image encoder, the prompt encoder and the mask decoder.

[0071] The image segmentation mask is filtered and screened to filter out non-circular and non-elliptical image segmentation masks, and a filtered image segmentation mask is obtained.

[0072] The filtered image segmentation mask is received and feature extracted with the text prompt to obtain M text feature vectors and image feature vectors, similarity calculation is performed on the M text feature vectors and image feature vectors to obtain M similarity scores, and the text feature vector with the largest similarity score with the image feature vector is selected as the classification result, thereby realizing instance segmentation of the celestial body crater image.

[0073] It should be noted that the text prompt is feature extracted by using a multi-modal image-text large model of CLIP, ALBEF and BLIP.

[0074] Specifically, the present application is further described below through embodiments:

[0075] The M text feature vectors and image feature vectors are similarity calculated to obtain the calculation process of the M similarity scores.

[0076] The similarity calculation is performed in a cosine similarity manner, and the probability mapping of classification comes from the Softmax processing of the similarity score, which is represented as:

[0077]

[0078] Where score i denotes the probability of the image segmentation mask corresponding to the i-th text, and here there are j classes, and there are j texts.

[0079] Embodiment two: in the second aspect, as Figure 3 shown, the present application discloses a celestial body crater image instance segmentation system based on a text prompt, comprising:

[0080] An image segmentation module is used to receive a celestial body crater image, perform image segmentation on the celestial body crater image, and obtain an image segmentation mask.

[0081] The filtering and screening module is configured to filter and screen the image segmentation mask, filter out non-circular and non-elliptical image segmentation masks, and obtain a filtered image segmentation mask.

[0082] The extraction calculation module is configured to receive the text prompt, perform feature extraction on the filtered image segmentation mask and the text prompt, obtain M text feature vectors and an image feature vector, perform similarity calculation on the M text feature vectors and the image feature vector, obtain M similarity scores, select a text feature vector with the largest similarity score with the image feature vector as a classification result, and realize instance segmentation of the celestial body crater image.

[0083] In combination with the second aspect, in some implementations of the second aspect, the system further includes: the celestial body crater image in the image segmentation module is obtained by a detector camera;

[0084] Or the image segmentation module performs image segmentation on the celestial body crater image by inputting the celestial body crater image into a pre-established SAM model for image segmentation.

[0085] Or the pre-established SAM model in the image segmentation module includes: an image encoder, a prompt encoder, and a mask decoder; the prompt encoder adopts a text prompt-based input mode

[0086] Or the expression of the pre-established SAM model in the image segmentation module is as follows:

[0087] F img = Φ i-enc (I)

[0088] T prompt = Φ p-enc ({p})

[0089] Masks = Φ m-dec (F img +F c-mask , T prompt )

[0090] I represents an input celestial body crater image, Φ i-enc represents an image encoder, F img represents intermediate features of the input crater image obtained by the image encoder. p represents a text prompt, Φ p-enc represents a prompt encoder, T prompt represents embedded tokens output by the prompt encoder from the text prompt, Φ m-dec represents a mask decoder, F c-mask represents an input mask, and Masks represents a result output by the mask decoder.

[0091] Preferably, the image segmentation module is used to segment the image of the celestial body meteor crater, and the segmented image is output from the mask decoder after being processed by the image encoder, the prompt encoder and the mask decoder.

[0092] Preferably, the mask screening module adopts a roundness screening method, that is, according to prior knowledge, the shape of most meteor craters meets the circular or elliptical shape. For the judgment of the circular shape, the output of the SAM model contains the pixel area S of the image segmentation mask, and the radius R can be obtained. Therefore, the circumference can be represented as The circumference of the image mask region output by the SAM can be represented as p, and when n=d / p is closer to 1, the shape region is more circular. Define θ=a / b, where a and b represent the long and short semi-axes of the ellipse, respectively. When n=θ=1, the shape is determined to be circular. For the judgment of the elliptical shape, the area of the ellipse can be represented as W=πab, and if the image segmentation mask shape is an ellipse, then q=W / S=1 is satisfied. Through the above constraints, the corresponding irregular and missegmented image segmentation masks can be effectively filtered out.

[0093] Preferably, the feature extraction of the text prompt in the extraction calculation module is performed by using a multi-modal image-text large model such as CLIP, ALBEF or BLIP.

[0094] Preferably, the calculation process of the M text feature vectors and the image feature vectors in the extraction calculation module to obtain M similarity scores is as follows:

[0095] The similarity is calculated by using the cosine similarity, and the probability mapping of classification comes from the Softmax processing of the similarity score, which is represented as:

[0096]

[0097] where score i represents the probability of the image segmentation mask corresponding to the i-th text, and there are j classes in total, and j texts.

[0098] Based on the same inventive concept, the present application further provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the program comprises program instructions, and the processor is configured to execute the program instructions stored in the memory. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are configured to implement one or more instructions, and are specifically configured to load and execute one or more instructions in the computer storage medium to implement the above method.

[0099] It needs to be further explained that, based on the same inventive concept, the present application further provides a computer storage medium, which stores a computer program, and the computer program is executed by the processor to perform the above method. The storage medium can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component.

[0100] In the description of the present application, the description of the terms "one embodiment", "an example", "a specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0101] The foregoing presents and describes the basic principles, main features and advantages of the present disclosure. It should be understood by those skilled in the art that the present disclosure is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, various changes and improvements can be made to the present disclosure, and all these changes and improvements fall within the scope of the present disclosure.

Claims

1. A text cue based celestial crater image instance segmentation method, characterized in that, The method comprises the following steps: Receiving a celestial body crater image, performing image segmentation on the celestial body crater image to obtain an image segmentation mask; The image segmentation on the celestial body crater image is performed by inputting the celestial body crater image into a pre-established SAM model for image segmentation; The pre-established SAM model comprises an image encoder, a prompt encoder and a mask decoder; the prompt encoder adopts a text prompt-based input mode; The expression of the pre-established SAM model is as follows: wherein, represents an input astronomical crater image, represents an image encoder, represents intermediate features of the input crater image obtained by the image encoder, represents a text prompt, represents a prompt encoder, represents embedded tokens output by the prompt encoder from the text prompt, represents a mask decoder, represents an input mask, represents a result output by the mask decoder; Filtering and screening the image segmentation mask to filter out non-circular and non-elliptical image segmentation masks to obtain a filtered image segmentation mask; The screening mode for filtering and screening the image segmentation mask adopts roundness screening: wherein the output of the SAM model contains the pixel area of the image segmentation mask , the radius is obtained , the circumference is expressed as , the image mask area circumference output by the SAM is expressed as , when is close to 1, the shape area is determined to be a circle; define , wherein a and b respectively represent the long semi-axis and the short semi-axis of the ellipse, when , the shape is determined to be a circle, for the judgment of the elliptical shape, the area of the ellipse is expressed as , if the image segmentation mask shape is an ellipse, it satisfies ; Receiving a text prompt, performing feature extraction on the filtered image segmentation mask and the text prompt to obtain M text feature vectors and image feature vectors, performing similarity calculation on the M text feature vectors and image feature vectors to obtain M similarity scores, selecting a text feature vector with the largest similarity score with the image feature vector as the classification result, and realizing instance segmentation on the celestial body crater image; The feature extraction on the text prompt is performed by using a multi-modal image-text large model such as CLIP, ALBEF or BLIP; The calculation process of the M text feature vectors and image feature vectors for similarity calculation to obtain M similarity scores is as follows: The similarity calculation is performed in a cosine similarity mode, and the probability mapping of classification comes from the Softmax processing of the similarity scores, which is expressed as: wherein denotes the probability that the image segmentation mask corresponds to the i-th text, where there are j classes in total, for j texts.

2. The method according to claim 1, wherein, The celestial body crater image is obtained by shooting with a detector camera.

3. The text cue-based celestial crater image instance segmentation method according to claim 2, characterized in that, The celestial body crater image is output from the mask decoder after passing through the image encoder, the prompt encoder and the mask decoder.

4. A text cue based celestial crater image instance segmentation system, characterized in that, It comprises: An image segmentation module for receiving a celestial body crater image, performing image segmentation on the celestial body crater image to obtain an image segmentation mask; The image segmentation on the celestial body crater image in the image segmentation module is performed by inputting the celestial body crater image into a pre-established SAM model for image segmentation; The pre-established SAM model in the image segmentation module comprises an image encoder, a prompt encoder and a mask decoder; the prompt encoder adopts a text prompt-based input mode The expression of the pre-established SAM in the image segmentation module is as follows: represents an input astronomical meteor crater image, represents an image encoder, represents intermediate features of the input meteor crater image after passing through the image encoder; represents a text prompt, represents a prompt encoder, represents embedded tokens of the text prompt after passing through the prompt encoder, represents a mask decoder, represents an input mask, represents a result after passing through the mask decoder; A filtering and screening module for filtering and screening the image segmentation mask to filter out non-circular and non-elliptical image segmentation masks to obtain a filtered image segmentation mask; The screening mode for filtering and screening the image segmentation mask in the filtering and screening module adopts roundness screening: wherein the output of the SAM model contains the pixel area of the image segmentation mask , the radius is obtained , the circumference is expressed as , the image mask area circumference output by the SAM is expressed as , when is close to 1, the shape area is determined to be a circle; define , wherein a and b respectively represent the long semi-axis and the short semi-axis of the ellipse, when , the shape is determined to be a circle, and for the judgment of the elliptical shape, the area of the ellipse is expressed as , if the image segmentation mask shape is an ellipse, then is satisfied; An extraction and calculation module for receiving a text prompt, performing feature extraction on the filtered image segmentation mask and the text prompt to obtain M text feature vectors and image feature vectors, performing similarity calculation on the M text feature vectors and image feature vectors to obtain M similarity scores, selecting a text feature vector with the largest similarity score with the image feature vector as the classification result, and realizing instance segmentation on the celestial body crater image; The text prompts in the extraction calculation module are used for feature extraction by using the multi-modal large models of CLIP, ALBEF and BLIP; In the extraction calculation module, the M text feature vectors and the image feature vectors are subjected to similarity calculation to obtain a calculation process of M similarity scores: The similarity calculation is performed in the cosine similarity mode, and the probability mapping of classification is obtained by Softmax processing of the similarity scores, which is represented as: wherein denotes the probability that the image segmentation mask corresponds to the i-th text, where there are j classes in total, for j texts.

5. The text cue based celestial crater image instance segmentation system according to claim 4, wherein, The celestial body meteor crater image is obtained by a detector camera in the image segmentation module. The celestial body meteor crater image in the image segmentation module is subjected to image encoding, prompt encoding and mask decoding, and an image segmentation mask is output from the mask decoder.

Images