An image segmentation method, system, device and storage medium

Abstract

The present invention provides an image segmentation method, system, device and storage medium, comprising the following steps: collecting and organizing thin slice images and obtaining the corresponding segmentation descriptions of the thin slices; using the thin slice images and the corresponding segmentation descriptions of the thin slices as training data sets, validation data sets and test data sets, and performing standardization processing; establishing an ap-SAM segmentation model based on the standardized training data sets, validation data sets and test data sets; optimizing the ap-SAM segmentation model by using the stochastic gradient descent method and iterating to the optimal ap-SAM segmentation model; performing test segmentation on the thin slice images in the test data set based on the optimal ap-SAM segmentation model, comparing the prediction results of the optimal ap-SAM segmentation model for the test data set with the standard labels, analyzing the prediction results, and obtaining the final thin slice segmentation model; this application has great potential in the segmentation of thin slice particles.

Description

Technical Field

[0001] The present invention belongs to the technical field of geological thin section analysis, and particularly relates to an image segmentation method, system, device and storage medium. Background Art

[0002] In the current era of continuous progress in sedimentary geology, advanced chemical analysis and element testing methods are emerging one after another. However, the traditional method of identifying and analyzing transparent minerals under a polarized light microscope still plays an indispensable role in the fields of mineralogy, petrology, ore deposit geology, geochemistry, structural geology, specific materials science, and environmental science. Thin section information statistics has always been an important part of geological research. Quickly and accurately obtaining information in thin sections has been an area of research constantly explored in the field of geological thin sections. Currently, the mainstream method is still to manually count the information in thin sections. Although this manual observation can provide reliable sample compositions and texture features, this process requires a large amount of manpower, is prone to errors, and requires professional knowledge.

[0003] With significant progress made in the substantial expansion of data and model scales in basic model research, these models usually use millions of data obtained from the network to train their structures with hundreds of millions of parameters, thereby generating representations with generalization and transfer capabilities. Therefore, pre-trained basic models usually only need to be simply fine-tuned or adjusted to be applied to various downstream tasks. In the field of vision, the large vision model SAM launched by META can achieve category-independent image segmentation. SAM was trained on a dataset with 11 million images and more than 1 billion masks. In addition, SAM also introduced an interactive prompt-based segmentation mode, significantly enhancing its zero-shot learning ability. In this mode, the model is provided with an image and some visual prompts, such as boxes, points, or masks, to indicate the regions in the image that the model needs to segment. This enables SAM to easily adapt to various specific downstream tasks, including the fields of medicine, industry, agriculture, and remote sensing. Therefore, an image segmentation method is expected.

[0004] We propose an automated point detection method that can automatically obtain the particle point prompts required by SAM according to different thin sections, so as to use SAM to obtain the particle segmentation results in the thin sections. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the present invention provides an image segmentation method, system, device and storage medium, which can automatically obtain the particle point prompts required by SAM according to different thin sections, so as to use SAM to obtain the particle segmentation results in the thin sections.

[0006] In a first aspect, an embodiment of the present invention provides an image segmentation method, including the following steps:

[0007] Collect and organize slice images and obtain the corresponding segmentation description of the slice;

[0008] The slice images and the corresponding segmentation descriptions of the slices are used as training datasets, validation datasets, and test datasets, and are standardized.

[0009] Establish an ap-SAM segmentation model based on the standardized training dataset, validation dataset, and test dataset;

[0010] The stochastic gradient descent method is used to optimize the ap-SAM segmentation model and iterate to the optimal ap-SAM segmentation model;

[0011] The thin slice images in the test data set are tested and segmented based on the optimal ap-SAM segmentation model. The prediction results of the optimal ap-SAM segmentation model for the test data set are compared with the standard labels. The prediction results are analyzed to obtain the final thin slice segmentation model.

[0012] Furthermore, when the slice image and the corresponding segmentation description of the slice are used as the training dataset, the verification dataset and the test dataset, the slice image is scaled to a size of 384x576 and divided into the training dataset, the verification dataset and the test dataset in a ratio of 6:2:2.

[0013] Furthermore, the process of the standardization process is:

[0014]

[0015] Where x norm is the standardized value, x is the input image pixel, the image pixel is in RGB form, mean and std are the mean and variance calculated on the training data set.

[0016] Furthermore, the ap-SAM segmentation model includes a coarse prediction mechanism and a fine-tuning mechanism. The coarse prediction mechanism adopts an inner circle detection module and a point sampling strategy, and the fine-tuning adopts a SAM segmentation model.

[0017] Furthermore, the coarse prediction mechanism corresponds to a coarse prediction stage during the process of iterating to the optimal ap-SAM segmentation model, and includes the following steps:

[0018] Inputting the slice image data information into the image encoder of the SAM segmentation model in the fine-tuning stage to obtain the intermediate layer high-dimensional feature information of the slice image data information;

[0019] The obtained high-dimensional feature information of the intermediate layer is input into the internal circle detection module to extract the particle information in the slice;

[0020] In the internal circle detection module, dimensionality reduction is performed on each intermediate-layer high-dimensional feature information, and the intermediate-layer high-dimensional feature information is used to extract the global context information of the data. The global context information is added to each intermediate-layer high-dimensional feature information;

[0021] All the intermediate-layer high-dimensional feature information is concatenated and feature-fused to obtain feature information I, where the ASPP (Atrous Spatial Pyramid Pooling) is used to concatenate and feature-fuse the intermediate-layer high-dimensional feature information;

[0022] The feature information I obtained in the internal circle detection module is sent to the center detection layer (Center layer) and the radius detection layer (Radius layer) to detect the center coordinates and radius;

[0023] According to the obtained center coordinate information, radius information, and point sampling strategy, a series of point prompt information required by SAM is sampled;

[0024] The fine-tuning mechanism corresponds to the fine-tuning stage in the process of iterating to the optimal ap-SAM segmentation model, including the following steps:

[0025] Based on the input thin slice image data information, the intermediate-layer high-dimensional feature information of the thin slice image data information is obtained through the image encoder of the SAM segmentation model;

[0026] The obtained intermediate-layer high-dimensional feature information of the thin slice image data and the point prompt information obtained through the rough prediction stage are sent to the remaining module of the SAM segmentation model to obtain the particle segmentation mask:

[0027] Furthermore, the point sampling strategy follows specific sampling principles, including the following steps:

[0028] According to the predicted inner circle radius r, the particles are divided into three size groups: large, medium, and small. For particles of different sizes, different scaling factors are used to cover the entire particle area with the selected points. For large particles, a larger scaling factor is selected; for small particles, a smaller scaling factor is used;

[0029] According to the obtained scaling factor, center coordinate information, and radius information, four points required for each particle are determined. The position information of these four points will be input into the SAM segmentation module for image segmentation. For the selection of the three size groups of large, medium, and small, when r is greater than 32, v is selected as 0.6; when r is less than 32 but greater than 16, v is selected as 0.7; when r is less than 16, v is selected as 0.8;

[0030] Points=(x±(r*v),y±(r*v));

[0031] Among them, v is the scaling factor, (x, y) represents the center coordinates of the inner circle predicted by the module, and r represents the radius of the predicted inner circle;

[0032] The process of feeding the intermediate high-dimensional feature information of the obtained thin slice image data and the point hint information obtained in the rough prediction stage into the remaining module of the SAM segmentation model to obtain the particle segmentation mask is as follows:

[0033] Mask = SAM mask_decoder (SAM prompt_encoder (Points), Y);

[0034] In the formula, Mask represents the thin slice particle segmentation result, SAM prompt_encoder and SAM mask_decoder respectively represent the prompt encoder and the mask decoder in SAM.

[0035] Furthermore, the optimizer used in the process of optimizing the ap-SAM segmentation model is AdamW; the loss function used in the random gradient descent method is the combined loss function of Focal loss and L1 loss; in the process of analyzing the prediction results, the performance ability of the optimal ap-SAM segmentation model in thin slice segmentation and the ability to predict the inner circle of particles are analyzed.

[0036] In the embodiment of the present invention, in the second aspect, the embodiment of the present invention provides an image segmentation system, including:

[0037] An acquisition module, configured to collect and organize thin slice images and obtain the segmentation description corresponding to the thin slices;

[0038] A preprocessing module, configured to use the thin slice images and the segmentation descriptions corresponding to the thin slices as training data sets, validation data sets, and test data sets, and perform normalization processing;

[0039] A model establishment module, configured to establish an ap-SAM segmentation model based on the normalized training data sets, validation data sets, and test data sets;

[0040] An optimization module, configured to optimize the ap-SAM segmentation model by using the random gradient descent method and iterate to the optimal ap-SAM segmentation model;

[0041] An output module, configured to perform test segmentation on the thin slice images in the test data set based on the optimal ap-SAM segmentation model, compare the prediction results of the optimal ap-SAM segmentation model on the test data set with the standard labels, analyze the prediction results, and obtain the final thin slice segmentation model.

[0042] In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned image segmentation method are implemented.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned image segmentation method are implemented.

[0044] Some other optional features and technical effects of the embodiments of the present invention are described below, and some can be understood by reading this article.

[0045] Compared with the prior art, the present invention has the following beneficial technical effects:

[0046] The present invention provides an image segmentation method, system, device and storage medium, including the following steps: collecting and organizing thin slice images and obtaining the segmentation descriptions corresponding to the thin slices; using the thin slice images and the segmentation descriptions corresponding to the thin slices as training data sets, validation data sets and test data sets, and performing normalization processing; establishing an ap-SAM segmentation model based on the normalized training data sets, validation data sets and test data sets; using the stochastic gradient descent method to optimize the ap-SAM segmentation model and iterating to the optimal ap-SAM segmentation model; based on the optimal ap-SAM segmentation model, performing test segmentation on the thin slice images in the test data set, comparing the prediction results of the optimal ap-SAM segmentation model for the test data set with the standard labels, and analyzing the prediction results to obtain the final thin slice segmentation model; the optimal ap-SAM segmentation model of the present application can automatically generate a series of point prompt information according to the possible particles in the input thin slice image to prompt and utilize SAM to segment the particles in the thin slice. On the basis of making full use of the excellent capabilities of the basic model SAM, a series of point prompts can be generated by predicting the internal circles of the particles and combining the point sampling strategy of the present invention, improving the accuracy of thin slice particle segmentation; the present application can achieve an AP accuracy of 49.2% on the test data set. In addition, the present invention can well explore the potential of the basic model SAM in the field of geological thin slice segmentation. Compared with other thin slice segmentation deep models, the present invention only needs to perform some simple geological thin slice adaptation to achieve good results on thin slice particles. Therefore, the present invention has great potential in thin slice particle segmentation. Description of the Drawings

[0047] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The elements shown are not limited by the proportions shown in the drawings. The same or similar reference numerals in the drawings represent the same or similar elements, where:

[0048] Figure 1 Schematic flowchart of an image segmentation method according to an embodiment of the present invention;

[0049] Figure 2 Overall architecture of the ap-SAM segmentation model according to an embodiment of the present invention;

[0050] Figure 3 Schematic flowchart of the method in the rough prediction stage according to an embodiment of the present invention;

[0051] Figure 4 Particle segmentation result of ap-SAM on the test dataset and prediction result of the internal circle of the particle according to an embodiment of the present invention;

[0052] Figure 5 An image segmentation system according to an embodiment of the present invention;

[0053] Figure 6 An image segmentation device according to the present invention.

[0054] In the figure: 900, segmentation system; 910, acquisition module; 920, preprocessing module; 930, model establishment module; 940, optimization module; 950, output module; 1000, electronic device; 1001, processor; 1002, read-only memory; 1003, memory; 1004, bus; 1005, I / O interface; 1006, input part; 1007, output part; 1008, storage part; 1009, communication part; 1010, driver; 1011, removable medium. Detailed implementation manners

[0055] To make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the specific implementation manners and the accompanying drawings. Herein, the illustrative implementation manners of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0056] The term "including" and its variations used herein mean open inclusion, that is, "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The term "one exemplary embodiment" and "one embodiment" mean "at least one exemplary embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0057] Figure 1The image segmentation method 100 according to an embodiment of the present invention is shown. At step S101, thin slice images are collected and sorted out to obtain the segmentation description corresponding to the thin slices. Next, at step S102, the thin slice images and the segmentation descriptions corresponding to the thin slices are used as training data sets, validation data sets, and test data sets, and are subjected to standardization processing. Next, at step S103, an ap-SAM (automatic measurement point SAM) segmentation model is established based on the standardized training data sets, validation data sets, and test data sets. Next, at step S104, the ap-SAM segmentation model is optimized using the stochastic gradient descent method, and iterated to the optimal ap-SAM segmentation model. Next, at step S105, the thin slice images in the test data set are tested and segmented based on the optimal ap-SAM segmentation model, the prediction results of the optimal ap-SAM segmentation model for the test data set are compared with the standard labels, and the prediction results are analyzed to obtain the final thin slice segmentation model.

[0058] In one embodiment, when using the thin slice images and the segmentation descriptions corresponding to the thin slices as training data sets, validation data sets, and test data sets, the thin slice images are scaled to a size of 384x576 and divided into training data sets, validation data sets, and test data sets according to a ratio of 6:2:2.

[0059] In one embodiment, the process of the standardization processing is as follows:

[0060]

[0061] where x norm is the value after standardization processing, x is the input image pixel, the image pixel is in RGB format, and mean and std are the mean and variance calculated on the training data set.

[0062] In one embodiment, the ap-SAM segmentation model includes a rough prediction mechanism and a fine-tuning mechanism. The rough prediction mechanism uses an internal circle detection module and a point sampling strategy, and the fine-tuning uses the SAM segmentation model.

[0063] In one embodiment, the internal circle detection module can be implemented using the following steps:

[0064] Preprocess the input image, for example, perform smoothing using Gaussian filtering; use Canny edge detection and Hough transform to detect circles from the smoothed image; at this time, more circles can be detected by adjusting the threshold of the Hough transform; binarize the original image to prepare for obtaining the points contained in each circle later; for each circle detected in the first step, obtain the ratio of the number of black points contained inside it to the area of the circle, and these ratios can reflect the condition of the circle; among all the obtained ratios, find the largest one, and the circle corresponding to it is taken as the final inner circle. It should be noted that those skilled in the art can make corresponding adjustments and optimizations according to specific requirements and image characteristics. In addition, according to specific application scenarios, more complex algorithms and techniques can be adopted to achieve more accurate inner circle detection.

[0065] In one embodiment, the rough prediction mechanism corresponds to the rough prediction stage during the iteration to the optimal ap-SAM segmentation model. Figure 3 Method 200 of the rough prediction stage is shown, as Figure 3 shown, at step S201, input the thin slice image data information into the image encoder of the SAM segmentation model in the fine-tuning stage to obtain the intermediate high-dimensional feature information of the thin slice image data information; next, at step S202, input the obtained intermediate high-dimensional feature information into the inner circle detection module to extract the particle information in the thin slice; next, at step S203, perform dimensionality reduction on each intermediate high-dimensional feature information in the inner circle detection module, and use the intermediate high-dimensional feature information to extract the global context information of the data, and add all the global context information to each intermediate high-dimensional feature information; next, at step S203, splice and fuse all the intermediate high-dimensional feature information to obtain feature information I, where the ASPP (Atrous Spatial Pyramid Pooling) is used to splice and fuse the intermediate high-dimensional feature information; next, at step S204, send the feature information I obtained in the inner circle detection module to the center detection layer Center layer and the radius detection layer Radius layer to detect the center coordinates and radius; next, at step S205, sample a series of point prompt information required by the SAM segmentation model according to the obtained center coordinate information, radius information, and point sampling strategy.

[0066] In one embodiment, at step S201, the process of inputting the thin slice image data information into the image encoder of the SAM segmentation model in the fine-tuning stage to obtain the intermediate high-dimensional feature information of the thin slice image data information is as follows:

[0067] Y,Y i =SAM image_encoder (X);

[0068] Wherein, represents the thin slice image data information, where B represents the batch size input to the network, C represents the dimension of the current feature map, H×W is the size of the current feature map, and Y represents the high-dimensional feature information obtained by encoding the input thin slice image data information through the image encoder. Y i represents the intermediate high-dimensional feature information during the processing of the image information by the image encoder, where i∈{1-4} is the layer marker of the model, and SAM image_encoder is the image encoder in SAM;

[0069] At step S202, the obtained intermediate high-dimensional feature information Y i is input into the internal circle detection module Circle_detection to extract the particle information in the thin slice:

[0070] (x,y),r=Circle_detection(Y i );

[0071] Wherein, (x,y) represents the center coordinates of the internal circle predicted by the module, r represents the radius of the predicted internal circle, and Circle_detection is the internal circle detection module;

[0072] At step S203, each Y in the internal circle detection module Circle_detection i needs to be first dimensionally reduced to reduce the computational pressure in the next step; and the global context information global_inf of the data is extracted using the intermediate high-dimensional feature information Y4, and the global context information is added to each Y i :

[0073] Y i =Down(Y i );

[0074] global_inf=Global(Y4);

[0075] Y i =Y i +global_inf;

[0076] Wherein, Down is the dimensional reduction module, and Global is the global context information extraction module;

[0077] At step S204, all Y i are concatenated and feature fused to obtain the feature information I. The process of using ASPP (Atrous Spatial Pyramid Pooling) to concatenate and feature fuse the intermediate high-dimensional feature information is:

[0078] I = Fusion(Cat(Y i ));

[0079] I = ASPP(I) + I;

[0080] Wherein, Cat is the splicing operation, and Fusion is the feature fusion module.

[0081] In step S204, the feature information I obtained in the internal circle detection module Circle_detection will be sent to the center detection layer Center layer and the radius detection layer Radius layer for detecting the center coordinates and radius. Both the center detection layer and the radius detection layer are composed of multiple convolutional layers:

[0082] (x, y) = Center(I);

[0083] r = Radius(I);

[0084] In step S205, according to the obtained center (x, y), radius information r, and the point sampling strategy, a series of point prompt information Points required by SAM will be sampled:

[0085] Points = Point_sample((x, y), r);

[0086] In the formula, Points is a series of sampled point prompt information, and Point_sample is the point sampling strategy;

[0087] The fine-tuning mechanism corresponds to the fine-tuning stage in the process of iterating to the optimal ap-SAM segmentation model, including the following steps:

[0088] Based on the input thin slice image data information, the intermediate high-dimensional feature information of the thin slice image data information is obtained through the image encoder of the SAM segmentation model;

[0089] The obtained intermediate high-dimensional feature information of the thin slice image data and the point prompt information obtained in the rough prediction stage are sent to the remaining module of the SAM segmentation model to obtain the particle segmentation mask:

[0090] Mask = SAM mask_decoder (SAM prompt_encoder (Points), Y);

[0091] In the formula, Mask represents the thin slice particle segmentation result, and SAM prompt_encoder and SAM mask_decoder represent the prompt encoder and the mask decoder in SAM respectively.

[0092] In one embodiment, the Point_sample point sampling strategy follows specific sampling principles, including the following steps:

[0093] According to the predicted inner circle radius r, the particles are divided into three size groups: large, medium, and small. For particles of different sizes, different scaling factors are used to ensure that the selected points better cover the entire particle area. For large particles, a larger scaling factor is selected to ensure that the selected points can completely cover the range of the particles; for small particles, a smaller scaling factor is used to avoid the selected points exceeding the boundaries of the particles.

[0094] Based on the obtained scaling factor, center coordinate information, and radius information, four points required for each particle are determined. The position information of these four points will be input into the SAM segmentation model for image segmentation. For the selection of the three size groups of large, medium, and small, when r is greater than 32, v is selected as 0.6; when r is less than 32 but greater than 16, v is selected as 0.7; when r is less than 16, v is selected as 0.8.

[0095] Points = (x ± (r * v), y ± (r * v));

[0096] where v is the scaling factor, (x, y) represents the center coordinates of the inner circle predicted by the module, and r represents the radius of the predicted inner circle.

[0097] In one embodiment, the optimizer adopted during the process of optimizing the ap-SAM segmentation model is AdamW; the loss function used in the random gradient descent method is a combined loss function of Focal loss and L1 loss.

[0098] In one embodiment, during the process of analyzing the prediction results, the performance of the optimal ap-SAM segmentation model in thin slice segmentation and the ability to predict the inner circle of particles are analyzed.

[0099] One embodiment provided by the present invention is as follows:

[0100] Collect river sediments from the Yarlung Zangbo River channel, then put the sandy sediments into epoxy resin, and then perform mechanical polishing to make thin slices with a thickness of 0.03 mm and a smooth and flat surface; use a standard polarized light microscope to collect thin slice images of the processed thin slices, and annotate and explain the thin slice images. The main annotation objects are the possible particles in the thin slices.

[0101] After collecting the thin slice images, a dataset is constructed using the thin slice image data. First, the thin slice images are scaled to a size of 384x576 for convenient processing by the model. Then, these thin slice image data are divided into a training dataset, a validation dataset, and a test dataset in a ratio of 6:2:2, and the data is normalized. The method is as follows:

[0102]

[0103] In the formula, x norm is the value after normalization, x is the pixel of the input image in RGB form, and mean and std are the mean and variance calculated on the training dataset.

[0104] Build the ap-SAM segmentation model, which is written using the Pytorch framework (version 1.10.0). As Figure 2 shown, Figure 2 shows the overall architecture of the ap-SAM segmentation model in this embodiment. The image editor module is responsible for processing the input image information, and part of the processed image information is input into the internal original detection module and the point sampling module for processing to complete Figure 2 (b) The rough prediction stage. The remaining processed image information and the results of the rough prediction stage are input into the mask decoder and the prompt encoder to complete Figure 2 (a) The fine-tuning stage.

[0105] Put the thin slice images into ap-SAM and iterate 32 times. Each time, calculate the loss using the predicted value and the true value of the model, and use the loss to optimize the model so that the predicted value of the model continuously approaches the true value. After the loss stabilizes, the trained optimal ap-SAM segmentation model can be obtained.

[0106] Send the constructed test dataset into the trained ap-SAM segmentation model for testing, and compare the true segmentation data of the test dataset with the segmentation data predicted by the model to calculate its accuracy.

[0107] Figure 4 Shows the segmentation effect of part of the optimal ap-SAM segmentation model in this embodiment on the thin slices of river sediments in the Yarlung Zangbo River channel and the prediction effect of the internal circles of particles. ap-SAM can reach an AP accuracy of 49.2% in the test dataset after 32 iterations. The solution of this embodiment can be applied to the segmentation of thin slices of river sediments in the Yarlung Zangbo River channel, improving the automatic segmentation ability of thin slice images.

[0108] In summary, using the ap-SAM deep learning model proposed by the present invention for automatic segmentation of thin slice images has high calculation efficiency and can obtain good segmentation accuracy on thin slice images.

[0109] Figure 5 FIG. 900 shows an image segmentation system according to an embodiment of the present invention, including:

[0110] The acquisition module 910 is configured to collect and organize thin slice images and obtain the segmentation description corresponding to the thin slices;

[0111] The preprocessing module 920 is configured to use the thin slice images and the segmentation descriptions corresponding to the thin slices as training data sets, validation data sets, and test data sets, and perform normalization processing;

[0112] The model establishment module 930 is configured to establish an ap-SAM segmentation model based on the normalized training data sets, validation data sets, and test data sets;

[0113] The optimization module 940 is configured to optimize the ap-SAM segmentation model by using the stochastic gradient descent method and iterate to the optimal ap-SAM segmentation model;

[0114] The output module 9:50 is configured to perform test segmentation on the thin slice images in the test data set based on the optimal ap-SAM segmentation model, compare the prediction results of the optimal ap-SAM segmentation model for the test data set with the standard labels, analyze the prediction results, and obtain the final thin slice segmentation model.

[0115] In an embodiment of the present invention, an electronic device is provided, including: a processor and a memory storing a computer program, and the processor is configured to execute the lithology identification method of any embodiment of the present invention when running the computer program.

[0116] Figure 6 FIG. 1000 shows a schematic diagram of an electronic device that can implement the method of an embodiment of the present invention or implement the electronic device of an embodiment of the present invention. In some embodiments, there may be more or fewer electronic devices than shown in the figure. In some embodiments, a single or multiple electronic devices may be used for implementation. In some embodiments, cloud or distributed electronic devices may be used for implementation.

[0117] As Figure 6As shown, the electronic device 1000 includes a processor 1001, which can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) 1002 or programs and / or data loaded from a storage section 1008 into a random access memory (RAM) 1003. The processor 1001 can be a multi-core processor or can include multiple processors. In some embodiments, the processor 1001 can include a general main processor and one or more special coprocessors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), and so on. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0118] The above-mentioned processor and memory are jointly used to execute the programs stored in the memory, and when the programs are executed by a computer, they can implement the methods, steps, or functions described in the above embodiments.

[0119] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, a touch screen, etc.; an output section 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read from it can be installed into the storage section 1008 as needed. Figure 6 Only some components are schematically shown, and it does not mean that the computer system 1000 only includes Figure 6 the components shown.

[0120] The systems, devices, modules, or units illustrated in the above embodiments can be implemented by a computer or its associated components. The computer can be, for example, a mobile terminal, a smart phone, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device, a smart TV, an Internet of Things system, a smart home, an industrial computer, a server, or a combination thereof.

[0121] Although not shown, in an embodiment of the present invention, a storage medium is provided, and the storage medium stores a computer program, and the computer program is configured to execute the compilation method based on file differences in any embodiment of the present invention when being run.

[0122] The storage medium in the embodiment of the present invention includes permanent and non-permanent, removable and non-removable articles that can implement information storage by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0123] The methods, programs, systems, devices, etc. in the embodiments of the present invention can be executed or implemented in a single or multiple networked computers, and can also be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be executed by remote processing devices connected through a communication network.

[0124] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, system, or computer program product. Therefore, those skilled in the art can conceive that the implementation of the functional modules / units or controllers and related method steps clarified in the above embodiments can be achieved in a manner combining software, hardware, and soft / hardware.

[0125] Unless explicitly stated, the actions or steps of the methods and programs described according to the embodiments of the present invention do not necessarily have to be executed in a specific order and can still achieve the desired results. In some embodiments, multi-tasking and parallel processing are also possible or may be advantageous.

[0126] In this article, multiple embodiments of the present invention are described. However, for the sake of brevity, the descriptions of each embodiment are not exhaustive, and the same or similar features or parts between the various embodiments may be omitted. In this article, "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" mean applicable to at least one embodiment or example according to the present invention, rather than all embodiments. The above terms do not necessarily mean referring to the same embodiment or example. Without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0127] Exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are only examples of the best mode for implementing the systems and methods. Those skilled in the art can understand that various changes can be made to the embodiments of the systems and methods described herein when implementing the systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. An image segmentation method, characterized in that, Including the following steps: Collect and organize the thin slice images and obtain the corresponding segmentation descriptions of the thin slices; Use the thin slice images and the corresponding segmentation descriptions of the thin slices as the training dataset, validation dataset, and test dataset, and perform normalization processing; Based on the normalized training dataset, validation dataset, and test dataset, establish an ap-SAM segmentation model. The ap-SAM segmentation model includes a coarse prediction mechanism and a fine-tuning mechanism. The coarse prediction mechanism uses an internal circle detection module and a point sampling strategy, and the fine-tuning mechanism uses a SAM segmentation model; Use the stochastic gradient descent method to optimize the ap-SAM segmentation model and iterate to the optimal ap-SAM segmentation model; Among them, during the process of iterating to the optimal ap-SAM segmentation model, the coarse prediction mechanism corresponds to the coarse prediction stage, including the following steps: Input the thin slice image data information into the image encoder of the SAM segmentation model in the fine-tuning stage to obtain the intermediate high-dimensional feature information of the thin slice image data information; Input the obtained intermediate high-dimensional feature information into the internal circle detection module to extract the particle information in the thin slice; Perform dimensionality reduction processing on each intermediate high-dimensional feature information in the internal circle detection module, and use the intermediate high-dimensional feature information to extract the global context information of the data, and add the global context information to each intermediate high-dimensional feature information; In the internal circle detection module, splice and fuse all the intermediate high-dimensional feature information to obtain feature information I, where the ASPP atrous spatial pyramid pooling is used to splice and fuse the intermediate high-dimensional feature information; Send the feature information I obtained in the internal circle detection module to the center detection layer Center layer and the radius detection layer Radius layer to detect the center coordinates and radius; According to the obtained center coordinate information, radius information, and the point sampling strategy, sample a series of point prompt information required by the SAM segmentation model; Among them, during the process of iterating to the optimal ap-SAM segmentation model, the fine-tuning mechanism corresponds to the fine-tuning stage, including the following steps: Based on the input thin slice image data information, obtain the intermediate high-dimensional feature information of the thin slice image data information through the image encoder of the SAM segmentation model; Send the obtained intermediate high-dimensional feature information of the thin slice image data and the point prompt information obtained through the coarse prediction stage to the remaining module of the SAM segmentation model to obtain the particle segmentation mask; Based on the optimal ap-SAM segmentation model, perform test segmentation on the thin slice images in the test dataset, compare the prediction results of the optimal ap-SAM segmentation model for the test dataset with the standard labels, analyze the prediction results, and obtain the final thin slice segmentation model.

2. The image segmentation method according to claim 1, wherein When using the thin slice images and the corresponding segmentation descriptions of the thin slices as the training dataset, validation dataset, and test dataset, scale the thin slice images to a size of 384x576 and divide them into the training dataset, validation dataset, and test dataset according to a ratio of 6:2:

2.

3. The image segmentation method according to claim 1, wherein The process of the normalization processing is: where x norm is the value after normalization, x is the input image pixel, the image pixel is in RGB format, and mean and std are the mean and variance calculated on the training dataset.

4. The image segmentation method according to claim 1, wherein The point sampling strategy follows specific sampling principles, including the following steps: According to the predicted inner circle radius r, the particles are divided into three size groups: large, medium, and small. For particles of different sizes, different scaling factors are used to cover the entire particle area with the selected points. Among them, for large particles, a larger scaling factor is selected, while for small particles, a smaller scaling factor is used; Four points required for each particle are determined based on the obtained scaling factor, center coordinate information, and radius information. The position information of these four points will be input into the SAM segmentation model for image segmentation. For the selection of the three size groups of large, medium, and small, when r is greater than 32, v is selected as 0.6, when r is less than 32 but greater than 16, v is selected as 0.7, and when r is less than 16, v is selected as 0.8; Points = (x ± (r * v), y ± (r * v)); Among them, v is the scaling factor, (x, y) represents the center coordinates of the inner circle predicted by the module, and r represents the radius of the predicted inner circle; The process of feeding the intermediate high-dimensional feature information of the obtained thin slice image data and the point prompt information obtained in the rough prediction stage into the remaining module of the SAM segmentation model to obtain the particle segmentation mask is as follows: Mask=SAM mask_decoder (SAM prompt_encoder (Points),Y); Where Mask represents the segmentation result of thin-film particles, SAM prompt_encoder and SAM mask_decoder represent the prompt encoder and the mask decoder in the SAM segmentation model respectively, and Y represents the high-dimensional feature information of the intermediate layer.

5. The image segmentation method according to claim 1, wherein The optimizer used in the process of optimizing the ap-SAM segmentation model is AdamW; The loss function used in the random gradient descent method is a combined loss function of Focal loss and L1 loss; In the process of analyzing the prediction results, the performance of the optimal ap-SAM segmentation model in thin slice segmentation and the ability to predict the inner circle of particles are analyzed.

6. An image segmentation system, characterized in that, An image segmentation method based on any one of claims 1-5 includes: An acquisition module for collecting and organizing thin slice images and obtaining the segmentation description corresponding to the thin slices; A preprocessing module for using the thin slice images and the segmentation descriptions corresponding to the thin slices as training data sets, validation data sets, and test data sets, and performing normalization processing; A model establishment module for establishing an ap-SAM segmentation model based on the normalized training data sets, validation data sets, and test data sets; An optimization module for optimizing the ap-SAM segmentation model using the random gradient descent method and iterating to the optimal ap-SAM segmentation model; An output module for performing test segmentation on the thin slice images in the test data set based on the optimal ap-SAM segmentation model, comparing the prediction results of the optimal ap-SAM segmentation model for the test data set with the standard labels, analyzing the prediction results, and obtaining the final thin slice segmentation model.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of an image segmentation method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of an image segmentation method as described in any one of claims 1-5.

Images