Genotyping methods and apparatuses, devices, and media for choroidal melanoma

Abstract

The application provides a choroidal melanoma genotyping method and device, equipment and medium, and belongs to the technical field of image processing. The method comprises the following steps: obtaining a target choroidal melanoma image, dividing the target choroidal melanoma image into regions, obtaining a plurality of target region sub-images, performing genotyping classification on each target region sub-image through a preset classifier, obtaining a first genotyping probability of each target region sub-image being classified as a target genotyping, performing weighted average on each first genotyping probability, obtaining a second genotyping probability of the target choroidal melanoma image being classified as the target genotyping, and if the second genotyping probability is greater than a preset threshold, classifying the target choroidal melanoma image as the target genotyping, so that the genotyping of the choroidal melanoma can be predicted.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and medium for genotyping choroidal melanoma. Background Technology

[0002] Choroidal melanoma is an ocular tumor. Traditional pathological analysis can provide information about the tumor's tissue structure and morphological characteristics, but it is difficult to provide detailed information about its genotype. Different genotypes of choroidal melanoma respond differently to therapeutic drugs, and the analysis and determination of genotypes are of great significance for the initial screening and treatment of choroidal melanoma. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, device, and medium for genotyping choroidal melanoma, with the aim of predicting the genotype of choroidal melanoma.

[0004] To achieve the above objectives, a first aspect of this application proposes a genotyping method for choroidal melanoma, the method comprising:

[0005] Acquire images of the target choroidal melanoma;

[0006] The target choroidal melanoma image is divided into regions to obtain multiple target region sub-images;

[0007] Each target region sub-image is classified into genotypes by a preset classifier to obtain the first genotype probability that each target region sub-image is classified into the target genotype.

[0008] The weighted average of each of the first genotype probabilities is used to obtain the second genotype probability that the target choroidal melanoma image is classified as the target genotype.

[0009] If the probability of the second genotype is greater than a preset threshold, the target choroidal melanoma image is classified as the target genotype.

[0010] In some embodiments, the process of dividing the target choroidal melanoma image into multiple target region sub-images includes:

[0011] The target choroidal melanoma image is divided into regions to obtain multiple initial region sub-images;

[0012] Obtain the grayscale histogram of each of the initial region sub-images;

[0013] For each of the initial region sub-images, the number of tumor pixels and the number of non-tumor pixels in the initial region sub-image are obtained according to the grayscale histogram, and the tumor coverage of the initial region sub-image is determined according to the number of tumor pixels and the number of non-tumor pixels.

[0014] Multiple initial region sub-images are filtered based on the tumor coverage to obtain multiple target region sub-images.

[0015] In some embodiments, the preset classifier includes a feature extraction module, a spatial attention module, and a fully connected layer. The feature extraction module includes multiple network layers, and the last three network layers of the feature extraction module are each connected in series with a spatial attention module. The step of performing genotyping classification on each target region sub-image using the preset classifier to obtain a first genotyping probability that each target region sub-image is classified as a target genotype includes:

[0016] For each target region sub-image, the feature extraction module performs feature extraction on the target region sub-image to obtain the preliminary region features of each of the last three network layers.

[0017] The preliminary region features are transformed by the spatial attention module connected in series with the network layer to obtain the candidate region features output by the spatial attention module.

[0018] The candidate region features output by each of the spatial attention modules are concatenated to obtain concatenated features;

[0019] The spliced ​​features are mapped by the fully connected layer to obtain the first genotype probability that each target region sub-image is classified as the target genotype.

[0020] In some embodiments, the feature extraction module includes multiple network layers, each including a standard convolutional block, multiple concatenated depthwise separable convolutional blocks, and an output block. The feature extraction module extracts features from the target region sub-image to obtain preliminary region features for each of the last three network layers, including:

[0021] The target region sub-image is subjected to a first convolutional process using the standard convolutional block to obtain the first convolutional image features;

[0022] The first convolutional image features are processed by a second convolution through multiple concatenated depthwise separable convolutional blocks to obtain the second convolutional image features output by each depthwise separable convolutional block.

[0023] The output image features are obtained by mapping the second convolutional image features output from the last depth-separable convolutional block through the output block;

[0024] The preliminary region features are obtained based on the output image features and multiple second convolutional image features.

[0025] In some embodiments, the second convolution processing of the first convolutional image features by concatenating multiple depthwise separable convolutional blocks to obtain the second convolutional image features output by each depthwise separable convolutional block includes:

[0026] The first convolutional image features are subjected to depthwise convolution to obtain the first candidate convolutional features;

[0027] The first candidate convolutional feature is subjected to point-by-point convolution processing to obtain the second candidate convolutional feature;

[0028] The first convolutional image features and the second candidate convolutional features are fused to obtain the second convolutional image features.

[0029] In some embodiments, the step of performing spatial attention transformation on the preliminary region features through the spatial attention module connected in series with the network layer to obtain candidate region features output by the spatial attention module includes:

[0030] Max pooling is performed on the preliminary region features to obtain the first pooled features;

[0031] The initial region features are averaged and pooled to obtain the second pooled features;

[0032] The first pooling feature and the second pooling feature are concatenated to obtain the first concatenated feature;

[0033] The first spliced ​​feature is convolved to obtain the second spliced ​​feature;

[0034] The second splicing feature is activated to obtain the candidate region feature.

[0035] In some embodiments, the preset classifier is trained according to the following steps:

[0036] Acquire images of choroidal melanoma samples; the choroidal melanoma images of the samples have genotyping tags;

[0037] Edge detection was performed on the choroidal melanoma images of the samples to obtain an edge point set;

[0038] The sample choroidal melanoma image is sampled according to the edge point set and the preset sampling rate to obtain multiple image blocks; the multiple image blocks share the same genotyping label;

[0039] Each image patch is input into an initial classifier for genotyping prediction to obtain the genotyping category of each image patch;

[0040] The target loss is obtained by calculating the loss based on the genotyping labels and genotyping categories of all the image patches;

[0041] The model parameters of the initial classifier are adjusted according to the target loss to obtain the preset classifier.

[0042] To achieve the above objectives, a second aspect of this application provides a genotyping device for choroidal melanoma, the device comprising:

[0043] The acquisition module is used to acquire images of the target choroidal melanoma.

[0044] The segmentation module is used to segment the target choroidal melanoma image into multiple target region sub-images.

[0045] The classification module is used to perform genotyping classification on each target region sub-image using a preset classifier, and to obtain the first genotyping probability that each target region sub-image is classified as the target genotyping.

[0046] The calculation module is used to perform a weighted average of the probabilities of each of the first genotypes to obtain the second genotype probability that the target choroidal melanoma image is classified as the target genotype;

[0047] The judgment module is used to classify the target choroidal melanoma image as the target genotype if the probability of the second genotype is greater than a preset threshold.

[0048] To achieve the above objectives, a third aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the gene typing method for choroidal melanoma described in the first aspect.

[0049] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the genotyping method for choroidal melanoma described in the first aspect.

[0050] This application proposes a genotyping method, device, electronic equipment, and computer-readable storage medium for choroidal melanoma. It acquires a target choroidal melanoma image and performs genotyping detection based on the pathological image. Tumor tissue information may be distributed across multiple regions of the target choroidal melanoma image. To fully extract pathological feature information, the target choroidal melanoma image is divided into multiple target region sub-images. A preset classifier is used to perform genotyping classification on each target region sub-image to fully mine feature information at different locations in the image, obtaining the first genotyping probability of each target region sub-image being classified as the target genotype. To avoid the influence of data randomness on genotyping prediction, a weighted average of the first genotyping probabilities is applied. This multi-region joint prediction significantly improves the accuracy of genotyping prediction, obtaining the second genotyping probability of the target choroidal melanoma image being classified as the target genotype. If the probability of the second genotype is greater than the preset threshold, the target choroidal melanoma image is classified as the target genotype, thus achieving accurate prediction of the genotype of choroidal melanoma. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the genotyping method for choroidal melanoma provided in the embodiments of this application;

[0052] Figure 2 This is a flowchart of the genotyping method for choroidal melanoma provided in the embodiments of this application;

[0053] Figure 3 yes Figure 2 The flowchart of step S220 in the text;

[0054] Figure 4 This is a schematic diagram of the division of choroidal melanoma images provided in the embodiments of this application;

[0055] Figure 5 This is a schematic diagram of the screening of choroidal melanoma images provided in an embodiment of this application;

[0056] Figure 6 yes Figure 2 The flowchart of step S230 in the text;

[0057] Figure 7 yes Figure 6 The flowchart of step S610 in the text;

[0058] Figure 8 yes Figure 7 The flowchart of step S720 in the process;

[0059] Figure 9 yes Figure 6 The flowchart of step S620 in the process;

[0060] Figure 10 This is a flowchart of the training process of the preset classifier provided in the embodiments of this application;

[0061] Figure 11 This is a heatmap of genotyping for choroidal melanoma provided in an embodiment of this application;

[0062] Figure 12 This is a schematic diagram of the genotyping device for choroidal melanoma provided in an embodiment of this application;

[0063] Figure 13 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0067] Choroidal melanoma is an ocular tumor. Traditional pathological analysis can provide information about the tumor's tissue structure and morphological characteristics, but it is difficult to provide detailed information about its genotype. Different genotypes of choroidal melanoma respond differently to therapeutic drugs, and the analysis and determination of genotypes are of great significance for the initial screening and treatment of choroidal melanoma.

[0068] With the development of deep learning technology in the field of medical image analysis, genotype-based classification prediction using pathological slide images has become possible. Deep learning technology can learn features and perform classification predictions from large-scale pathological image data, providing strong support for personalized treatment, early diagnosis, disease mechanism research, and clinical decision-making. However, there is still a significant research gap in the classification prediction of choroidal melanoma genotyping, especially algorithms based on pathological slides.

[0069] Based on this, embodiments of this application provide a genotyping method, a genotyping device, an electronic device, and a computer-readable storage medium for choroidal melanoma, with the aim of predicting the genotype of choroidal melanoma.

[0070] The gene typing method, gene typing device, electronic device, and computer-readable storage medium for choroidal melanoma provided in this application are specifically described through the following embodiments. First, the gene typing method for choroidal melanoma in this application embodiment is described.

[0071] The genotyping method for choroidal melanoma provided in this application relates to the field of image processing technology. This genotyping method for choroidal melanoma can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the genotyping method for choroidal melanoma, but is not limited to the above forms.

[0072] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0073] Figure 1This is a schematic diagram of the genotyping method for choroidal melanoma provided in this application embodiment. Genotyping involves the mutation and expression model of specific genes in tumor cells. Genotyping can be BP1 genotyping, KIT genotyping, etc. A choroidal melanoma image is acquired, and region bounding is performed on the choroidal melanoma image to obtain multiple image patches. These multiple image patches are input into a classifier for genotyping prediction to obtain the genotyping of each image patch. Based on the joint decision of the genotyping of each image patch, the genotyping of the choroidal melanoma image is determined. The classifier is a binary classifier used to determine whether an image patch belongs to the target genotyping, which is a specific genotyping. If the image patch belongs to the target genotyping, the classifier outputs 1; otherwise, the classifier outputs 0.

[0074] Figure 2 This is an optional flowchart of the genotyping method for choroidal melanoma provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S210 to S250.

[0075] Step S210: Obtain an image of the target choroidal melanoma;

[0076] Step S220: Divide the target choroidal melanoma image into regions to obtain multiple target region sub-images;

[0077] Step S230: Perform genotyping classification on each target region sub-image using a preset classifier to obtain the first genotyping probability of each target region sub-image being classified as the target genotyping.

[0078] Step S240: The weighted average of each first genotype probability is calculated to obtain the second genotype probability that the target choroidal melanoma image is classified as the target genotype.

[0079] Step S250: If the probability of the second gene typing is greater than the preset threshold, the target choroidal melanoma image is classified as the target gene typing.

[0080] In step S210 of some embodiments, a target choroidal melanoma image is obtained using ophthalmic imaging technology or from a choroidal melanoma genotyping dataset. The target choroidal melanoma image is an image of choroidal melanoma with an undetermined genotype, or a pathological section image of a patient with choroidal melanoma. The choroidal melanoma genotyping dataset can be a publicly available dataset such as the TCGA research database, or a self-created dataset containing multiple choroidal melanoma images. To ensure information security, the choroidal melanoma genotyping dataset needs to be anonymized. It should be noted that the publicly available dataset in the TCGA research database contains 80 samples with two genotypes. When training the genotyping prediction model, the training and test sets can be divided on a sample-by-sample basis. The ratio of the training to the test set is 8:2, meaning 80% of the total data is used as the training set, and the remaining 20% ​​as the test set. Each pathological slide sample has a genotyping label, with the two genotyping labels being 0 and 1, respectively. There are 35 samples of type 1 and 45 samples of type 0, and the label distribution is relatively even.

[0081] Please see Figure 3 In some embodiments, step S220 may include, but is not limited to, steps S310 to S340:

[0082] Step S310: Divide the target choroidal melanoma image into regions to obtain multiple initial region sub-images;

[0083] Step S320: Obtain the grayscale histogram of each initial region sub-image;

[0084] Step S330: For each initial region sub-image, obtain the number of tumor pixels and the number of non-tumor pixels in the initial region sub-image based on the gray-level histogram, and determine the tumor coverage of the initial region sub-image based on the number of tumor pixels and the number of non-tumor pixels.

[0085] Step S340: Filter multiple initial region sub-images based on tumor coverage to obtain multiple target region sub-images.

[0086] In step S310 of some embodiments, tumor tissue information may be distributed at various locations in the target choroidal melanoma image. To facilitate the model's better understanding of the image content and improve the accuracy of genotyping prediction, the target choroidal melanoma image is non-overlappingly cropped to divide the image into multiple grid regions, resulting in multiple initial region sub-images. This allows for the full extraction of local features, yielding finer-grained tumor tissue information at different locations. Furthermore, compared to the entire image, region partitioning reduces computational complexity and improves the efficiency of genotyping prediction. The initial region sub-images are pathological slide image patches obtained from region partitioning. Figure 4 As shown, each initial region sub-image can be an image patch of equal size, with each patch measuring 512×512. Larger image patches contain more contextual information, enabling a more comprehensive capture of the features of choroidal melanoma, such as morphological and structural features. Smaller image patches focus more on local details of choroidal melanoma, such as color and hardness features. To accommodate choroidal melanomas of different sizes and locations, each initial region sub-image can also be an image patch of different sizes.

[0087] In step S320 of some embodiments, if the initial region sub-image is a color image, it is converted into a grayscale image. The grayscale levels are determined to be 0 to 255, for a total of 256 grayscale levels. For each grayscale level, the number of pixels in the initial region sub-image with that grayscale level is calculated. A grayscale histogram is plotted with the grayscale levels as the horizontal axis and the number of pixels with that grayscale level as the vertical axis.

[0088] In step S330 of some embodiments, after completing the non-overlapping cropping of the tumor region slices, some pathological slice blocks contain too little tumor tissue information, which may interfere with the learning of the algorithm model. Pathological slice blocks are then filtered based on grayscale histograms, and the statistical information of the grayscale histograms is used to remove pathological slices with insufficient foreground information. The initial region sub-image includes two types of pixels: foreground pixels and background pixels. Foreground pixels are tumor pixels, and background pixels are non-tumor pixels. For each initial region sub-image, the number of tumor pixels in the tumor pixels and the number of non-tumor pixels in the non-tumor pixels are obtained according to the grayscale histogram, and the tumor coverage of the initial region sub-image is determined based on the number of tumor pixels and the number of non-tumor pixels. The tumor coverage is the proportion of tumor pixels in the initial region sub-image (foreground proportion) or the proportion of non-tumor pixels in the initial region sub-image (background proportion). The sum of the number of tumor pixels and the number of non-tumor pixels equals the total number of pixels in the initial region sub-image. The formula for calculating the background proportion is:

[0089]

[0090] Where, p b p is the number of pixels that make up the background in the image. f The foreground component is the number of pixels in the image, while the background component is the ratio of the total number of background pixels to the total number of pixels in the image.

[0091] In step S340 of some embodiments, tumor coverage is determined by background proportion. A background proportion threshold is set. If the tumor coverage is less than the background proportion threshold, it indicates that the initial region sub-image contains a large number of tumor pixels, and the initial region sub-image is retained. If the tumor coverage is greater than or equal to the background proportion threshold, it indicates that the initial region sub-image contains a large number of non-tumor pixels, and the initial region sub-image is deleted, resulting in multiple target region sub-images. The background proportion threshold can be 0.2. Tumor coverage is determined by foreground proportion. A foreground proportion threshold is set. If the tumor coverage is greater than or equal to the foreground proportion threshold, the initial region sub-image is retained. If the tumor coverage is less than the foreground proportion threshold, the initial region sub-image is deleted, resulting in multiple target region sub-images. The foreground proportion threshold can be 0.8. Figure 5 As shown, the first image is a sub-image of the target region, and the other two images have been deleted.

[0092] Through steps S310 to S340, pathological slides containing tumor information can be screened out, avoiding the influence of irrelevant information on genotyping prediction. This improves both the accuracy and efficiency of genotyping prediction.

[0093] Each target region sub-image is input into a preset classifier for genotyping. The preset classifier can use a fixed-size input image (512×512) or an input image of any size. It's understood that the preset classifier can also predict multiple genotypes simultaneously. To achieve classification prediction for multiple genotypes, simply ensure that the number of output channels in the last linear layer of the classifier matches the number of classification categories; that is, the number of classification categories in the classifier should match the number of genotype categories.

[0094] Please see Figure 6 In some embodiments, the preset classifier includes a feature extraction module, a spatial attention module, and a fully connected layer. The feature extraction module includes multiple network layers, and the last three network layers of the feature extraction module are all connected in series with a spatial attention module. Step S230 may include, but is not limited to, steps S610 to S640:

[0095] Step S610: For each target region sub-image, the feature extraction module is used to extract features from the target region sub-image to obtain the preliminary region features of each of the last three network layers.

[0096] Step S620: The preliminary region features are transformed by spatial attention through the spatial attention module connected in series with the network layer to obtain the candidate region features output by the spatial attention module.

[0097] Step S630: The candidate region features output by each spatial attention module are concatenated to obtain concatenated features;

[0098] Step S640: The spliced ​​features are mapped through a fully connected layer to obtain the first genotyping probability of each target region sub-image being classified as the target genotype.

[0099] In step S610 of some embodiments, the feature extraction module may consist of mainstream classification models, such as ResNet50, ResNet101, ViT, Swin, and ConvNext. The feature extraction module has a hierarchical structure, with each network layer learning features at different levels. Shallow network layers learn low-level features such as edges, colors, textures, and shapes, while deeper network layers learn high-level features such as object shapes and scene types. To capture feature representations from low to high levels, this application selects the output features of the last three network layers to obtain preliminary region features. These preliminary region features are used to characterize the detailed features of the target region sub-image, and may include texture, corner points, tumor location, and tumor morphology.

[0100] In step S620 of some embodiments, since the lesion information contained in the pathological slices is widely distributed, this application introduces a spatial attention module to enhance the attention to and extraction of features distributed over a wide spatial range, thereby achieving a deeper level of image understanding. The connection between the feature extraction module and the spatial attention module is multi-level, that is, the last three network layers of the feature extraction module are connected in series with three spatial attention modules, with each network layer corresponding to one spatial attention module. The spatial attention module can be an STN model (Spatial Transformer Network). By performing spatial attention transformation on the preliminary region features output by the network layer through the spatial attention module connected in series with the network layer, candidate region features output by the spatial attention module are obtained.

[0101] In step S630 of some embodiments, the candidate region features output by the three spatial attention modules are concatenated from the channel dimension to obtain concatenated features.

[0102] In step S640 of some embodiments, the spliced ​​features are input to a fully connected layer, and feature mapping is performed on the spliced ​​features through the fully connected layer to obtain the final classification prediction output result, that is, the first genotype probability that each target region sub-image is classified as the target genotype. If the target genotype is BAP1, then the first genotype probability is the probability that the target region sub-image is classified as BAP1.

[0103] Through the above steps S610 to S640, tumor characteristic information of different regions can be fully captured, so as to perform genotyping prediction based on the tumor characteristic information of different regions, thereby improving the accuracy of genotyping prediction.

[0104] Please see Figure 7 In some embodiments, ConvNext-base is selected as the feature extraction module of the preset classifier, and multiple network layers include standard convolutional blocks, multiple depthwise separable convolutional blocks in series, and an output block. Step S610 may include, but is not limited to, steps S710 to S740:

[0105] Step S710: Perform a first convolution process on the target region sub-image using a standard convolution block to obtain the first convolution image features;

[0106] Step S720: Perform a second convolution process on the first convolutional image features by concatenating multiple depthwise separable convolutional blocks to obtain the second convolutional image features output by each depthwise separable convolutional block.

[0107] Step S730: The output image features are obtained by output mapping the second convolutional image features output by the last depth-separable convolutional block through the output block;

[0108] Step S740: Based on the output image features and multiple second convolutional image features, preliminary region features are obtained.

[0109] In step S710 of some embodiments, the standard convolutional block includes a two-dimensional convolutional layer and layer normalization. The target region sub-image is subjected to two-dimensional convolution through the two-dimensional convolutional layer, and the features obtained from the two-dimensional convolution are normalized to obtain the first convolutional image features.

[0110] In step S720 of some embodiments, the number of depthwise separable convolutional blocks is four. The first depthwise separable convolutional block consists of ConvNeXt blocks, and the remaining three depthwise separable convolutional blocks each consist of a downsampling layer and ConvNeXt blocks. The number of feature channels in the ConvNeXt blocks of the next depthwise separable convolutional block is twice the number of feature channels in the ConvNeXt blocks of the current depthwise separable convolutional block, and the number of feature channels is the same among multiple ConvNeXt blocks of the same depthwise separable convolutional block. The downsampling layer includes layer normalization and a two-dimensional convolutional layer. The first, second, and fourth depthwise separable convolutional blocks each include three ConvNeXt blocks connected in sequence, and the third depthwise separable convolutional block includes nine ConvNeXt blocks connected in sequence. The first convolutional image features are processed by the first depthwise separable convolutional block to obtain the second convolutional image features output by the first depthwise separable convolutional block. The second convolutional image features output from the current depth-separable convolutional block are input into the next depth-separable convolutional block. The above steps are repeated until the second convolutional image features output from the fourth depth-separable convolutional block are obtained, resulting in multiple second convolutional image features.

[0111] In step S730 of some embodiments, the output block includes a global average pooling layer, a layer normalization layer, and a linear layer. The second convolutional image features output from the last depthwise separable convolutional block are globally average pooled using the global average pooling layer to obtain global pooled features. These global pooled features are then normalized using the layer normalization layer, and the normalized features are linearly mapped using the linear layer to obtain the output image features.

[0112] In step S740 of some embodiments, the second convolutional image features and output image features output from the third and fourth depth-separable convolutional blocks are used as the preliminary region features of the last three network layers in sequence.

[0113] Through steps S710 to S740, key feature information for genotyping prediction can be extracted, avoiding interference from irrelevant feature information in the prediction process.

[0114] Please see Figure 8 In some embodiments, step S720 may include, but is not limited to, steps S810 to S830:

[0115] Step S810: Perform depthwise convolution on the first convolutional image features to obtain the first candidate convolutional features;

[0116] Step S820: Perform point-by-point convolution processing on the first candidate convolutional features to obtain the second candidate convolutional features;

[0117] Step S830: Perform feature fusion on the first convolutional image features and the second candidate convolutional features to obtain the second convolutional image features.

[0118] In step S810 of some embodiments, the ConvNeXt block uses a deep convolutional block. The main difference between a deep convolutional block and a standard convolutional block lies in how the channels are processed. A deep convolutional block independently convolves each channel and concatenates them, while a standard convolutional block sums the results of all channel convolutions and activates the sum using an activation function. Standard convolutional blocks enhance the feature representation capability of the feature extraction block, while deep convolutional blocks reduce the number of model parameters and improve computational efficiency. Combining the two not only improves the feature extraction capability of the preset classifier for different regions of tumor tissue information but also enhances the efficiency of tumor tissue information extraction.

[0119] The first depthwise separable convolutional block consists only of a ConvNeXt block. Taking the first depthwise separable convolutional block as an example, the extraction process of the second convolutional image features is described in detail. The features of the first convolutional image are processed by depthwise two-dimensional convolution through a depthwise convolutional block, and the features obtained by convolution are normalized by layer to obtain the first candidate convolutional features.

[0120] In step S820 of some embodiments, the first candidate convolutional features undergo a first pointwise two-dimensional convolution process, the features after the first pointwise two-dimensional convolution are subjected to GELU activation, the activated features undergo a second pointwise two-dimensional convolution process, the features after the second pointwise two-dimensional convolution are subjected to layer scaling, and the scaled features are regularized using path discarding to obtain the second candidate convolutional features. Here, pointwise convolution refers to performing two-dimensional convolution on the input using standard convolutional blocks. Layer scaling refers to multiplying the input tensors of different channels by different values ​​to refine and refine the features. Path discarding is a regularization technique that removes multi-branch structures in the model.

[0121] In step S830 of some embodiments, the first convolutional image features and the second candidate convolutional features are summed to obtain the second convolutional image features output by the first depthwise separable convolutional block. The processing procedures of the second, third, and fourth depthwise separable convolutional blocks involving the ConvNeXt block can all refer to the first depthwise separable convolutional block, and will not be repeated here.

[0122] In steps S810 to S830 above, each depthwise separable convolutional block uses local connectivity for feature fusion, which can capture global information of the image while maintaining computational efficiency, and avoid the reduction in genotyping prediction accuracy due to insufficient information expression.

[0123] Please see Figure 9In some embodiments, step S620 may include, but is not limited to, steps S910 to S950:

[0124] Step S910: Max pooling is performed on the preliminary region features to obtain the first pooled features;

[0125] Step S920: Perform average pooling on the preliminary region features to obtain the second pooled features;

[0126] Step S930: Concatenate the first pooling feature and the second pooling feature to obtain the first concatenated feature;

[0127] Step S940: Perform convolution processing on the first concatenated feature to obtain the second concatenated feature;

[0128] Step S950: Activate the second splicing feature to obtain candidate region features.

[0129] In step S910 of some embodiments, the spatial attention module consists of a max-pooling layer, an average-pooling layer, a convolutional layer, and a sigmoid activation function. The max-pooling layer performs max-pooling on the preliminary region features along the channel dimension, compressing the number of feature channels of the preliminary region features to 1, resulting in the first pooled feature. If the preliminary region features are a feature map with dimensions B×C×W×H, then the first pooled feature is a feature map with dimensions B×1×W×H. Here, B represents the batch size, C represents the number of feature channels, W represents the feature width, and H represents the feature height.

[0130] In step S920 of some embodiments, the preliminary region features are average pooled in the channel dimension by an average pooling layer, compressing the number of feature channels of the preliminary region features to 1, thus obtaining the second pooled feature. If the preliminary region features are a feature map with dimensions B×C×W×H, then the second pooled feature is a feature map with dimensions B×1×W×H.

[0131] In step S930 of some embodiments, the first pooling feature and the second pooling feature are concatenated along the channel dimension to obtain a first concatenated feature. The first concatenated feature is a feature map with dimensions B×2×W×H.

[0132] In step S940 of some embodiments, the first concatenated feature is convolved by a convolutional layer to obtain the second concatenated feature. The convolutional layer has 1 output channel and a kernel size of 7.

[0133] In step S950 of some embodiments, the second splicing feature is activated by the sigmoid activation function to obtain candidate region features, which are the output feature maps of the spatial attention module.

[0134] Steps S910 to S950 above, by employing a spatial attention mechanism to analyze choroidal melanoma images in the spatial dimension, can more accurately focus on important regions in the images, improve the model's feature selection ability, and thus achieve accurate prediction of genotyping.

[0135] From the training set data of the constructed choroidal melanoma genotyping dataset, sample choroidal melanoma images and their genotyping labels are obtained. Genotyping labels characterize the genotyping category of the sample choroidal melanoma image. To achieve data augmentation and adapt the input images to the input size during classifier training, data preprocessing operations such as cropping and filtering are performed on the sample choroidal melanoma images in the training set data. The specific data preprocessing process is as follows: Experienced ophthalmologists identify the full pathological slides (sample choroidal melanoma images), manually select and crop the pathological slides of the tumor region, and save them. The cropped pathological slides of the tumor region are further cropped into non-overlapping pathological slide blocks of a preset size, such as 512×512. Pathological slide blocks are filtered based on grayscale histograms. After the above series of data preprocessing operations, the training set data is cropped into 16,000 pathological slide blocks. The cropped data is further divided into a classifier training set and a classifier test set, with a ratio of 8:2. To prevent data leakage, the division of the classifier training set and the classifier test set is also done on a sample-by-sample basis. The classifier is trained using the training set and tested using the test set. During the training phase, the training set is input into the initial classifier for genotyping detection, obtaining the predicted probability of belonging to the genotyping label from the initial classifier's output. Based on the binary cross-entropy loss function (BCE), the training loss is calculated according to the predicted probabilities and the genotyping label. The initial classifier is trained by minimizing the training loss, resulting in the trained initial classifier. The binary cross-entropy loss function is expressed as:

[0136]

[0137] Among them, L BCE The value represents the loss function of the binary cross-entropy loss function; N is the number of samples in the classifier training set; i represents the i-th sample; y i p(y) represents the true label of the i-th sample; i ) indicates that the genotype category output by the classifier belongs to y. i The predicted probability of the label.

[0138] It should be noted that the classifier was trained on a workbench equipped with an NVIDIA-RTX3090 GPU, using Python and the PyTorch deep learning framework. The batch size was set to 128 during training, the Adam optimizer was used, and the training consisted of 100 epochs with an initial learning rate of 10. -4 The learning rate is adjusted using cosine annealing, and the termination learning rate is 10. -6 Minimize the training loss to adjust the initial classifier's model parameters. Cosine annealing is a method for adjusting the learning rate. During training, the learning rate is gradually decreased to allow for finer parameter adjustments as the model approaches convergence. Starting with the initial learning rate, it is adjusted according to the curvature of the cosine function, with a final termination learning rate of 10. -6 .

[0139] To improve the accuracy of the classifier in predicting genotypes, the initial classifier was further trained using a classifier test set.

[0140] Please see Figure 10 In some embodiments, the training process of the preset classifier may include, but is not limited to, steps S1010 to S1060:

[0141] Step S1010: Obtain an image of the choroidal melanoma sample; the choroidal melanoma image of the sample has a genotyping label;

[0142] Step S1020: Perform edge detection on the sample choroidal melanoma image to obtain an edge point set;

[0143] Step S1030: Sample the choroidal melanoma image of the sample according to the edge point set and the preset sampling rate to obtain multiple image blocks; multiple image blocks share the same genotyping label;

[0144] Step S1040: Input each image patch into the initial classifier for genotyping prediction to obtain the genotyping category of each image patch;

[0145] Step S1050: Calculate the loss based on the genotyping labels and genotyping categories of all image patches to obtain the target loss;

[0146] Step S1060: Adjust the model parameters of the initial classifier according to the target loss to obtain the preset classifier.

[0147] In step S1010 of some embodiments, sample choroidal melanoma images and their genotyping labels are obtained from the constructed classifier test set. The genotyping labels are used to characterize the genotyping category to which the sample choroidal melanoma images belong.

[0148] In step S1020 of some embodiments, the Sobe L operator is used to perform horizontal and vertical convolution operations on the sample choroidal melanoma image to detect horizontal and vertical edges in the image, obtaining an edge point set. The edge point set includes multiple pixels constituting the edge. Specifically, the horizontal and vertical grayscale changes of the image are differentiated by the horizontal and vertical Sobe L operators to obtain the horizontal gradient and vertical gradient. The total image gradient is determined based on the horizontal and vertical gradients, and the gradient direction is determined based on the horizontal and vertical gradients. Non-maximum suppression is applied to the total image gradient and gradient direction to mark the edge pixels in the image as edge points, thereby obtaining the edge points of the tumor region in the pathological image.

[0149] The horizontal Sobe L operator is shown below:

[0150]

[0151] The vertical Sobe L operator is shown below:

[0152]

[0153] The formulas for calculating the total gradient G and gradient direction θ of an image are as follows:

[0154]

[0155]

[0156] In step S1030 of some embodiments, the preset sampling rate represents the number of images sampled from the sample choroidal melanoma image. The preset sampling rate can be 5, 7, or 9. The edge point set is filtered to obtain the edge point P1(i1, j1) closest to the top left corner of the image, the edge point P2(i2, j2) closest to the top right corner of the image, the edge point P3(i3, j3) closest to the bottom left corner of the image, and the edge point P4(i4, j4) closest to the bottom right corner of the image. Based on the positions of these four edge points and the preset sampling rate, the image block region positions are calculated to randomly sample the sample choroidal melanoma image, obtaining multiple image blocks for data augmentation to increase the data volume of the sample image. Multiple image blocks share the same genotyping label, and each image block has the same input size, for example, 512×512. If the preset sampling rate is 5, then the region positions of the five image blocks can be represented as:

[0157] Patch1[i1:i1+512,j1:j1+512]

[0158] Patch2[i2-512:i2,j2:j2+512]

[0159] Patch3[i3:i3+512,j3-512:j3]

[0160] Patch4[i4-512:i4,j4-512:j4]

[0161]

[0162] The calculation method for the location of areas with preset sampling rates of 7 and 9 can be referred to the above calculation method, and will not be repeated here.

[0163] In step S1040 of some embodiments, each image patch is input to an initial classifier trained on a classifier training set for genotyping prediction to obtain the genotyping category of each image patch.

[0164] In step S1050 of some embodiments, the target loss is obtained by calculating the loss based on the genotyping labels and genotyping categories of all image patches according to the binary cross-entropy loss function.

[0165] In step S1060 of some embodiments, the initial classifier is trained by minimizing the target loss to obtain a preset classifier. Four different baseline models, resnet50, resnet101, swin-base, and convnext-base, are selected as feature extraction modules in the classifier, and their performance on the classifier test set is shown in the table below.

[0166] Resnet50 78.75 69.18 37.27 39.1 Resnet101 88.55 72.04 60.80 44.85 Swin-base 85.81 75.95 54.50 52.2 ConvNext-base 91.48 81.06 71.97 62.38

[0167] Table 1

[0168] As shown in Table 1, the classifier exhibits the highest classification performance when convnext-base is used as the feature extraction module, achieving a classification accuracy of 81.06% for a single pathological slide. It should be noted that the classification accuracy can be adjusted by modifying the preset sampling rate.

[0169] In addition to the comparative ablation experiments on the feature extraction module, ablation experiments were also conducted on the spatial attention module. The evaluation metric was the area under the ROC curve. The experimental results are shown in the table below:

[0170]

[0171] Table 2

[0172] As shown in Table 2, the area under the ROC curve of different feature extraction modules was improved after the spatial attention module was added, thus proving that the spatial attention module has the effect of improving classification performance.

[0173] Through the above steps S1010 to S1060, a classifier can be trained to predict the genotype of choroidal melanoma.

[0174] In step S240 of some embodiments, after the classifier independently predicts the genotype of multiple pathological slices, a weighted average is performed on the genotype probabilities obtained by the classifier for pathological slices at different locations. A joint decision is then made to obtain a second genotype probability that the target choroidal melanoma image is classified as the target genotype. The second genotype probability is expressed as:

[0175]

[0176] Among them, P o P represents the probability of the second genotype, n is the number of image patches, and P is the probability of the second genotype. j is the first genotype probability output by the classifier after the j-th image patch is processed.

[0177] In step S250 of some embodiments, if the probability of the second genotype is greater than a preset threshold, the target choroidal melanoma image is classified as the target genotype. The preset threshold can be 0.5. If the probability of the second genotype is less than or equal to the preset threshold, the target choroidal melanoma image is classified as a non-target genotype.

[0178] This application has performed inference performance verification on the test set and ablation experiments on the preset sampling rate n. The experimental results are shown in the table below.

[0179]

[0180] Table 3

[0181] As shown in Table 3, the accuracy was significantly improved by using the multi-region joint decision-making method for different feature extraction modules, with convnextbase showing an accuracy improvement of nearly 10 percentage points. When the number of samples was 5, 7, and 9, the best-performing feature extraction module was a classifier using ConvNextbase combined with a spatial attention module. Heatmaps of pathological sections with different genotypes are shown below. Figure 11 As shown, from Figure 11 This shows the feature regions that the classifier model focuses on when performing genotyping classification.

[0182] This application also provides a genotyping method for choroidal melanoma, including: acquiring a target choroidal melanoma image; dividing the target choroidal melanoma image into regions to obtain multiple target region sub-images; determining a preset number of classifiers based on the number of target region sub-images; performing genotyping classification on the target region sub-images in parallel using the preset number of classifiers; obtaining a first genotyping probability for each target region sub-image to be classified as the target genotype; one target region sub-image corresponds to one classifier; and performing genotyping prediction in parallel using multiple classifiers can improve the efficiency of genotyping prediction. Alternatively, determining the image position of the target region sub-image within the target choroidal melanoma image; grouping the multiple target region sub-images according to the image position to determine multiple groups; performing genotyping prediction on the groups according to the classifier corresponding to each group; obtaining a first genotyping probability for the target region sub-images of each group; and improving the prediction efficiency by assigning different classifiers to image patches at different positions, thereby improving the accuracy of genotyping prediction for image patches at different positions. The classifiers corresponding to different positions are trained based on the image patches at the corresponding positions. The weighted average of the probabilities of each primary genotype is used to obtain the probability of the secondary genotype for classifying the target choroidal melanoma image as the target genotype. If the probability of the secondary genotype is greater than a preset threshold, the target choroidal melanoma image is classified as the target genotype.

[0183] In some embodiments, if the classifier is a multi-class classifier, the classifier performs genotyping prediction on the target region sub-image to obtain the first genotyping probability of the target region sub-image belonging to each genotyping category. For the same genotyping category, the first genotyping probabilities of each target region sub-image are weighted and averaged, or the first genotyping probabilities are weighted and summed according to the weights of the target region sub-images to obtain the second genotyping probability of the target choroidal melanoma image being classified into that genotyping category. The weights of the target region sub-images can be obtained from the reciprocal of the image's entropy. For the second genotyping probabilities of each genotyping category, the genotyping category with the highest second genotyping probability is selected as the target genotyping.

[0184] In some embodiments, the process of obtaining multiple target region sub-images includes: detecting the tumor location in a target choroidal melanoma image to obtain the location region of the choroidal melanoma; acquiring an image of the location region of the choroidal melanoma from the target choroidal melanoma image to obtain a target image; dividing the target image into regions to obtain multiple target region sub-images; and adding pre-selected image patches from the target image to the target region sub-images. The pre-selected image patches can be image patches from areas with multiple choroidal melanoma occurrences. Location detection can pinpoint regions containing tumor tissue information, increasing the amount of feature data for effective regions in genotyping prediction, thereby improving prediction accuracy.

[0185] Please see Figure 12 This application also provides a genotyping device for choroidal melanoma, which can implement the above-mentioned genotyping method for choroidal melanoma. The genotyping device for choroidal melanoma includes:

[0186] The acquisition module 1210 is used to acquire images of the target choroidal melanoma.

[0187] The segmentation module 1220 is used to segment the target choroidal melanoma image into multiple target region sub-images.

[0188] The classification module 1230 is used to perform genotyping classification on each target region sub-image using a preset classifier, and obtain the first genotyping probability of each target region sub-image being classified as the target genotyping.

[0189] The calculation module 1240 is used to perform a weighted average of the probabilities of each first genotype to obtain the second genotype probability that the target choroidal melanoma image is classified as the target genotype.

[0190] The judgment module 1250 is used to classify the target choroidal melanoma image as the target genotype if the probability of the second genotype is greater than a preset threshold.

[0191] The specific implementation of this choroidal melanoma genotyping device is basically the same as the specific implementation of the choroidal melanoma genotyping method described above, and will not be repeated here.

[0192] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described genotyping method for choroidal melanoma. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0193] Please see Figure 13 , Figure 13 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0194] The processor 1310 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0195] The memory 1320 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1320 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1320 and is called and executed by the processor 1310 to execute the genotyping method for choroidal melanoma according to the embodiments of this application.

[0196] The input / output interface 1330 is used to implement information input and output;

[0197] The communication interface 1340 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0198] Bus 1350 transmits information between various components of the device (e.g., processor 1310, memory 1320, input / output interface 1330, and communication interface 1340);

[0199] The processor 1310, memory 1320, input / output interface 1330 and communication interface 1340 are connected to each other within the device via bus 1350.

[0200] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described genotyping method for choroidal melanoma.

[0201] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0202] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0203] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0204] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0205] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0206] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0207] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for genotyping choroidal melanoma, characterized by, The method includes: Acquire images of the target choroidal melanoma; The target choroidal melanoma image is divided into regions to obtain multiple target region sub-images; Each target region sub-image is classified into genotypes by a preset classifier to obtain the first genotype probability that each target region sub-image is classified into the target genotype. The weighted average of each of the first genotype probabilities is used to obtain the second genotype probability that the target choroidal melanoma image is classified as the target genotype. If the probability of the second genotype is greater than a preset threshold, the target choroidal melanoma image is classified as the target genotype. The preset classifier includes a feature extraction module, a spatial attention module, and a fully connected layer. The feature extraction module includes multiple network layers, each including a standard convolutional block, multiple concatenated depthwise separable convolutional blocks, and an output block. The last three network layers of the feature extraction module are each concatenated with a spatial attention module. The process of performing genotyping classification on each target region sub-image using the preset classifier to obtain the first genotyping probability that each target region sub-image is classified as a target genotype includes: For each target region sub-image, the feature extraction module extracts features from the target region sub-image to obtain preliminary region features for each of the last three network layers; the spatial attention module, connected in series with the network layer, performs spatial attention transformation on the preliminary region features to obtain candidate region features output by the spatial attention module; the candidate region features output by each of the spatial attention modules are concatenated to obtain concatenated features; the fully connected layer performs feature mapping on the concatenated features to obtain the probability that each target region sub-image is classified as the first genotype of the target genotype.

2. The method of genotyping choroidal melanoma according to claim 1, wherein, The process of dividing the target choroidal melanoma image into multiple target region sub-images includes: The target choroidal melanoma image is divided into regions to obtain multiple initial region sub-images; Obtain the grayscale histogram of each of the initial region sub-images; For each of the initial region sub-images, the number of tumor pixels and the number of non-tumor pixels in the initial region sub-image are obtained according to the gray-level histogram, and the tumor coverage of the initial region sub-image is determined according to the number of tumor pixels and the number of non-tumor pixels. Multiple initial region sub-images are filtered based on the tumor coverage to obtain multiple target region sub-images.

3. The method of genotyping choroidal melanoma according to claim 1, wherein, The step of extracting features from the target region sub-image using the feature extraction module to obtain preliminary region features for each of the last three network layers includes: The target region sub-image is subjected to a first convolutional process using the standard convolutional block to obtain the first convolutional image features; The first convolutional image features are processed by a second convolution through multiple concatenated depthwise separable convolutional blocks to obtain the second convolutional image features output by each depthwise separable convolutional block. The output image features are obtained by mapping the second convolutional image features output from the last depth-separable convolutional block through the output block; The preliminary region features are obtained based on the output image features and multiple second convolutional image features.

4. The method of genotyping choroidal melanoma according to claim 3, wherein, The second convolution process, which involves performing a second convolution on the first convolutional image features using multiple concatenated depthwise separable convolutional blocks to obtain the second convolutional image features output by each depthwise separable convolutional block, includes: The first convolutional image features are subjected to depthwise convolution to obtain the first candidate convolutional features; The first candidate convolutional feature is subjected to point-by-point convolution processing to obtain the second candidate convolutional feature; The first convolutional image features and the second candidate convolutional features are fused to obtain the second convolutional image features.

5. The method of genotyping choroidal melanoma according to claim 1, wherein, The step of performing spatial attention transformation on the preliminary region features through the spatial attention module connected in series with the network layer to obtain candidate region features output by the spatial attention module includes: Max pooling is performed on the preliminary region features to obtain the first pooled features; The initial region features are averaged and pooled to obtain the second pooled features; The first pooling feature and the second pooling feature are concatenated to obtain the first concatenated feature; The first spliced ​​feature is convolved to obtain the second spliced ​​feature; The second splicing feature is activated to obtain the candidate region feature.

6. The method of genotyping choroidal melanoma according to any one of claims 1 to 5, characterized in that, The preset classifier is trained according to the following steps: Acquire images of choroidal melanoma samples; the choroidal melanoma images of the samples have genotyping tags; Edge detection was performed on the choroidal melanoma images of the samples to obtain an edge point set; Based on the edge point set and the preset sampling rate, the sample choroidal melanoma image is sampled to obtain multiple image blocks; Multiple image patches share the same genotyping tag; Each image patch is input into an initial classifier for genotyping prediction to obtain the genotyping category of each image patch; The target loss is obtained by calculating the loss based on the genotyping labels and genotyping categories of all the image patches; The model parameters of the initial classifier are adjusted according to the target loss to obtain the preset classifier.

7. A genotyping device for choroidal melanoma, characterized in that The device includes: The acquisition module is used to acquire images of the target choroidal melanoma. The segmentation module is used to segment the target choroidal melanoma image into multiple target region sub-images. The classification module is used to perform genotyping classification on each target region sub-image using a preset classifier, and to obtain the first genotyping probability that each target region sub-image is classified as the target genotyping. The calculation module is used to perform a weighted average of the probabilities of each of the first genotypes to obtain the second genotype probability that the target choroidal melanoma image is classified as the target genotype; The judgment module is used to classify the target choroidal melanoma image as the target genotype if the probability of the second genotype is greater than a preset threshold. The preset classifier includes a feature extraction module, a spatial attention module, and a fully connected layer. The feature extraction module includes multiple network layers, each including a standard convolutional block, multiple concatenated depthwise separable convolutional blocks, and an output block. The last three network layers of the feature extraction module are each concatenated with a spatial attention module. The process of performing genotyping classification on each target region sub-image using the preset classifier to obtain the first genotyping probability that each target region sub-image is classified as a target genotype includes: For each target region sub-image, the feature extraction module extracts features from the target region sub-image to obtain preliminary region features for each of the last three network layers; the spatial attention module, connected in series with the network layer, performs spatial attention transformation on the preliminary region features to obtain candidate region features output by the spatial attention module; the candidate region features output by each of the spatial attention modules are concatenated to obtain concatenated features; the fully connected layer performs feature mapping on the concatenated features to obtain the probability that each target region sub-image is classified as the first genotype of the target genotype.

8. An electronic device, characterized by The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the gene typing method for choroidal melanoma as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program is executed by the processor, it implements the gene typing method for choroidal melanoma as described in any one of claims 1 to 6.

Images