A colonoscopy image detection method, device and electronic equipment

Abstract

The present invention belongs to the field of image detection technology and provides a colonoscopy image detection method, device and electronic device. The colonoscopy image detection method includes: inputting the colonoscopy image into a classification model to obtain a classification result; the classification model includes: a square feature extractor for extracting multiple grid feature vectors from the colonoscopy image; a ring template processing module for dividing the colonoscopy image into multiple ring areas based on the ring template, the ring template includes multiple concentric rings, and the valid rings and invalid rings in the multiple concentric rings are alternately distributed, and a concentric ring area is defined as an annular area; a ring feature extractor for obtaining multiple ring feature vectors; an encoder for encoding multiple grid feature vectors, multiple ring feature vectors and classification labels to obtain classification label feature vectors; a classifier for classifying the classification label feature vector to obtain a classification result. The present invention explicitly models the annular topological structure of the colonoscopy image, thereby improving the accuracy of colonoscopy image classification.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a colonoscopy image detection method, device and electronic equipment. Background Art

[0002] Colonoscopic images generally refer to images of the intestines captured using an intestinal endoscope (also known as a colonoscope). Computer vision technology can be used to classify colonoscopic images and aid in the diagnosis of intestinal diseases. For example, it can distinguish between ulcerative colitis (UC) and Crohn's disease (CD), and can also grade the pathological severity of UC (0, 1, 2, and 3). Accurately extracting key distinguishing features from colonoscopic images is crucial for accurate classification.

[0003] Related technologies utilize a masked autoencoder (MAE) self-supervised learning method based on mask reconstruction to train encoders and decoders, enabling the encoder to accurately extract key identifying features from colonoscopy images. However, these techniques typically employ random block or grid masks when designing masks, failing to consider the inherent characteristics of colonoscopy images. The inventors, through analysis and summary of a large number of colonoscopy images, discovered the following inherent characteristics of colonoscopy images:

[0004] 1. Depth of field gradient characteristics, that is, the radial characteristic characteristics of clear proximal mucosa and attenuated distal brightness, presents a clear annular topological structure, which can be compared Figure 2 Example observations of the colonoscopy images presented;

[0005] 2. Lesion-sensitive areas in colonoscopy images (such as Crohn's lesion-sensitive areas) often show annular structural destruction. It is difficult to capture such morphological clues using existing block or grid masks, such as Figure 9 Sample 1 image in ;

[0006] The masks used in related technologies are difficult to capture the inherent characteristics of colonoscopy images, resulting in the key identification features extracted by the encoder being less sensitive to lesion features, and the classification accuracy needs to be improved. Summary of the Invention

[0007] The present application aims to at least solve the technical problems existing in the prior art and provide a colonoscopy image detection method, device and electronic equipment.

[0008] In a first aspect, the present application provides a colonoscopy image detection method, comprising: obtaining a colonoscopy image; inputting the colonoscopy image into a classification model to obtain a classification result; wherein the classification model comprises: a square feature extractor for extracting multiple grid feature vectors from the colonoscopy image; a ring template processing module for dividing the colonoscopy image into multiple ring areas based on the ring template, wherein the ring template comprises multiple concentric rings, wherein valid rings and invalid rings are alternately distributed in the multiple concentric rings, and a concentric ring area is defined as an annular area; an annular feature extractor for obtaining multiple annular feature vectors corresponding to the multiple annular areas; an encoder for encoding the multiple grid feature vectors, the multiple annular feature vectors and the classification labels to obtain the classification label feature vector; and a classifier for classifying the classification label feature vector to obtain the classification result.

[0009] In a second aspect, the present application provides a colonoscopy image detection device, which is used to implement the colonoscopy image detection method provided in the first aspect of the present application, including: an image acquisition module, which acquires a colonoscopy image; a classification module, which inputs the colonoscopy image into a classification model to obtain a classification result; wherein the classification model includes: a square feature extractor, which is used to extract multiple grid feature vectors from the colonoscopy image; a ring template processing module, which divides the colonoscopy image into multiple ring areas based on the ring template, wherein the ring template includes multiple concentric rings, and the valid rings and invalid rings in the multiple concentric rings are alternately distributed, and a concentric ring area is defined as an annular area; an annular feature extractor, which obtains multiple annular feature vectors corresponding to the multiple annular areas; an encoder, which encodes the multiple grid feature vectors, the multiple annular feature vectors and the classification labels to obtain the classification label feature vector; a classifier, which classifies the classification label feature vector to obtain the classification result.

[0010] The beneficial technical effects of the colonoscopy image detection method and device provided by this application are:

[0011] Taking into account the inherent characteristics of colonoscopic images, multiple annular areas of the colonoscopic image are obtained through an annular template, which realizes the explicit modeling of the annular topological structure of the colonoscopic image, and extracts the annular feature vectors corresponding to each annular area respectively. This not only realizes the extraction of annular feature vectors according to the depth of field gradient, but also enables the annular feature vectors to focus on the radial structural continuity of the intestinal cavity, enhances the sensitivity to pathological features such as annular fold destruction, and simultaneously extracts multiple grid feature vectors of the colonoscopic image to obtain the spatial distribution information of the colonoscopic image. The encoder is used to encode the multiple grid feature vectors, multiple annular feature vectors and classification labels so that the classification label feature vector has significant category identifiability, thereby improving the accuracy of colonoscopic image classification.

[0012] On the third aspect, the present application provides a mask-based self-supervised learning method, including: constructing a sample pair set, wherein a sample pair is composed of two colonoscopy images randomly selected from a colonoscopy image set; using a checkerboard template to generate a spatially decoupled view and a phase-reversed view for each sample pair, and using the phase-reversed view as the reconstruction target of the sample pair; using a ring template to generate a ring-fused view including multiple ring areas for each sample pair; constructing a self-supervised learning model, and training the self-supervised learning model using the spatially decoupled view, phase-reversed view and ring-fused view of the sample pair set, wherein the self-supervised learning model includes: a square feature extractor, for extracting multiple grid feature vectors from the spatially decoupled view of the sample pair; a ring feature extractor, for obtaining multiple ring feature vectors corresponding to multiple ring areas of the ring-fused view of the sample pair; an encoder, for encoding multiple grid feature vectors and multiple ring feature vectors to obtain encoding features; a decoder, for reconstructing the reconstruction target based on the encoding features after adding a learnable mask token to obtain a reconstructed image.

[0013] The beneficial technical effects of the mask-based self-supervised learning method provided in this application are:

[0014] Traditional self-supervised algorithms only perform random mask reconstruction on a single colonoscopy image and learn the local feature associations within a single colonoscopy image, but ignore the high similarity of colonoscopy images across cases, resulting in the problem that the category discriminability of the encoder output features is not high. This application introduces a dual-sample random mask reconstruction mechanism, randomly selects two colonoscopy images from the unlabeled colonoscopy image set to construct a sample pair, and generates cross-sample synthetic views (spatial decoupling views, phase-reversed views, and annular fusion views). The synthetic views of the sample pairs are used to train the self-supervised learning model, which can achieve cross-case feature migration and local-global feature collaboration. The decoder uses the radial information leaked by the annular template and combines the cross-case knowledge of the two samples of the sample pair to achieve more accurate image reconstruction. Compared with the random mask of the related MAE technology, this method uses the annular structure prior constraint to make the reconstruction process more consistent with the anatomical characteristics of the intestinal cavity, thereby improving the category discriminability of the encoder output features in self-supervised learning.

[0015] Cross-case feature transfer leverages cross-case similarities in colonoscopy images (e.g., restricted color distribution and annular topology) to force the self-supervised learning model to learn common features (e.g., annular continuity of healthy mucosa) and local differences (e.g., morphological disruption of lesion areas) between cases through two-sample mask reconstruction, significantly improving the self-supervised learning model's sensitivity to subtle pathological changes.

[0016] Local-global feature collaboration refers to obtaining three synthetic views through mask operations of checkerboard templates and ring templates, which requires the self-supervised learning model to simultaneously reconstruct local areas from different samples, forcing the encoder to establish semantic associations across images (such as the contrast between ulcer areas and healthy mucosa), thereby enhancing the discriminability of feature expression.

[0017] In the fourth aspect, the present application provides a classification model training method, including: a self-supervised pre-training stage, executing the steps of the mask-based self-supervised learning method provided in the third aspect of the present invention; a supervised fine-tuning stage, executing: using a ring template processing module, a classifier, and a square feature extractor, a ring feature extractor and an encoder obtained at the end of the self-supervised pre-training stage to form a classification model; and training the classification model using a colonoscopy image classification sample set.

[0018] The beneficial technical effects of the classification model training method provided in this application are: in addition to the beneficial effects of the mask-based self-supervised learning method provided in the third aspect, it also has the beneficial technical effects of accurately classifying colonoscopic images and realizing accurate auxiliary diagnosis of intestinal diseases (such as distinguishing between ulcerative colitis (UC) and Crohn's disease (CD), such as grading the severity of ulcerative colitis (UC)).

[0019] In a fifth aspect, the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method provided in the first aspect, the third aspect, or the fourth aspect of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 1 is a flow chart of the colonoscopy image detection method provided in Example 1 of the present invention;

[0021] Figure 2 is an example of a colonoscopy image;

[0022] Figure 3 is a schematic structural diagram of the classification model in Example 1 of the present invention;

[0023] Figure 4 1 is a schematic structural diagram of a square feature extractor in Example 1 of the present invention;

[0024] Figure 5 1 is a schematic structural diagram of the ring feature extractor in Example 1 of the present invention;

[0025] Figure 6 2 is a schematic structural diagram of a depth of field layer processing module in Example 1 of the present invention;

[0026] Figure 7 1 is a schematic diagram of the encoder structure in Example 1 of the present invention;

[0027] Figure 8 2 is a schematic structural diagram of a colonoscopy image detection device provided in Example 2 of the present invention;

[0028] Figure 9 3 is a process diagram of the self-supervised learning method provided by Example 3 of the present invention;

[0029] Figure 10 Schematic diagram of the classification model training method provided in Example 4 of the present invention;

[0030] Figure 11 It is a structural diagram of an electronic device provided in Example 5 of the present invention. DETAILED DESCRIPTION

[0031] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0032] Example 1

[0033] The present embodiment provides a colonoscopic image detection method, the execution subject of the method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the colonoscopic image detection method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0034] In a preferred implementation of this embodiment, the flow chart of the colonoscopy image detection method is as follows: Figure 1 Shown, including:

[0035] Step A1: Obtain a colonoscopy image.

[0036] In this preferred embodiment, the colonoscopic image is an image of the intestine taken by an intestinal endoscope. Figure 2The example of a colonoscopy image is shown. Its inner wall wrinkles have a circular topology, and its color distribution is highly restricted to a narrow red-yellow-white color gamut. The execution subject is not limited to obtaining the colonoscopy image to be tested by establishing a communication link with the intestinal endoscope; alternatively, the colonoscopy image to be tested may be pre-stored in memory, and the execution subject may read the colonoscopy image from the memory.

[0037] Step A2: Input the colonoscopy image into the classification model to obtain the classification result.

[0038] like Figure 3 As shown, the classification model includes:

[0039] A square feature extractor is used to extract multiple grid feature vectors from colonoscopy images;

[0040] An annular template processing module divides the colonoscopy image into multiple annular regions based on an annular template, wherein the annular template includes multiple concentric rings, wherein valid rings and invalid rings are alternately distributed in the multiple concentric rings, and a concentric ring region is defined as an annular region;

[0041] A ring feature extractor, which obtains multiple ring feature vectors corresponding to multiple ring areas;

[0042] An encoder encodes the multiple grid feature vectors, the multiple ring feature vectors and the classification labels to obtain a classification label feature vector;

[0043] The classifier performs classification processing on the classification mark feature vector to obtain the classification result.

[0044] In this preferred embodiment, please refer to Figure 4 As shown in Figure 2, the square feature extractor includes a convolution layer and a linear mapping module connected in sequence, wherein the linear mapping module includes one or more cascaded linear mapping layers. The convolution kernel size of the convolution layer is , the step length is , perform zero-filling convolution operation, divide the colonoscopy image into multiple pixel blocks, and obtain the convolution features of each pixel block. The convolution features of each pixel block are then enhanced through the linear mapping module to obtain the grid feature vector corresponding to each pixel block.

[0045] For example, the size of the colonoscopy image is 384*384, the convolution kernel size of the convolution layer is 32×32, and the stride is 32. After the convolution operation, the colonoscopy image is divided into 36 16×16 pixel blocks, and the convolution features of each pixel block are obtained. The convolution features of each pixel block are represented by a 256-dimensional vector. The convolution features of each pixel block use a shared linear mapping module to increase the channel dimension to 768. Please refer to Figure 4In this example, the linear mapping module includes a cascaded first linear mapping layer (linear mapping layer 1) and a second linear mapping layer (linear mapping layer 2), and a ReLU activation function is set in the first linear mapping layer. Finally, 36 grid feature vectors are obtained, that is, 36 grid tokens, which can be used as token sequences. express, The colonoscopy image The grid feature vector of the pixel block.

[0046] from Figure 2 It can be seen that, constrained by the physical imaging mechanism of the annular wide-angle lens, the colonoscopic image exhibits the inherent characteristics of mapping from a three-dimensional cavity to a two-dimensional plane. First, lens distortion causes barrel-shaped distortion at the edge of the colonoscopic image, forming a central radial geometric deformation; second, the depth of field gradient effect is significant under macro imaging. The proximal mucosa presents a clear texture due to sufficient illumination, while the distal tissue produces brightness decay with increasing distance, forming concentric dark ring features; more importantly, the annular folds of the healthy intestinal wall form a topological structure with rotational symmetry in the two-dimensional projection, namely an annular topological structure, while lesions (such as the longitudinal ulcers of Crohn's disease) will destroy this spatial continuity, forming important morphological diagnostic clues, such as Figure 9 Based on this, the present application explicitly models the annular topology of colonoscopy images and constructs an annular template to improve the sensitivity to lesion features.

[0047] In this preferred embodiment, the annular template includes multiple concentric rings, in which valid rings and invalid rings are alternately distributed. The valid rings represent the parts that need to be extracted in the masking operation, and the invalid rings represent the parts that do not need to be extracted in the masking operation. A concentric ring area is defined as an annular area. The size of the annular template is the same as the colonoscopic image, so the outer edge of the outermost concentric ring is rectangular or square, and the inner edge is circular. The number and width of the concentric rings in the annular template can be adjusted according to the actual size of the colonoscopic image. Figure 5 An example of a ring template is given. In this example, the ring template includes 7 concentric rings, and defines white concentric rings as valid rings and black concentric rings as invalid rings. The outer→inner part is white-black alternating, and there are 4 valid rings.

[0048] In this preferred embodiment, in order to quickly extract multiple annular regions from the colonoscopy image, preferably, the annular template processing module performs:

[0049] In step C21, a mask operation is performed on the colonoscopy image using the annular template to obtain a plurality of first annular regions. Specifically, after the mask operation, the first annular regions corresponding to all valid rings are obtained, that is, a plurality of first annular regions are obtained.

[0050] Step C22, using a phase-reversal annular template that forms a radial complementary pair with the annular template to perform a mask operation on the colonoscopy image to obtain multiple second annular areas; the phase-reversal annular template includes multiple concentric rings with the same number as the concentric rings of the annular template, and the valid rings and invalid rings are alternately distributed in the multiple concentric rings, and a concentric ring area is defined as an annular area, and the concentric rings at the corresponding positions of the valid rings of the annular template in the phase-reversal annular template are invalid rings, and the concentric rings at the corresponding positions of the invalid rings of the annular template in the phase-reversal annular template are valid rings. Figure 5 A phase-reversal ring template example is given. It consists of seven concentric rings, with white concentric rings defined as valid rings and black concentric rings as invalid rings. The outer and inner parts of the template alternate between black and white, with three valid rings. Using the phase-reversal ring template, the annular regions corresponding to the invalid rings in the template can be extracted.

[0051] In step C23, a plurality of first annular areas and a plurality of second annular areas are combined into a plurality of annular areas. Figure 5 In the example given, there are 4 first annular areas and 3 second annular areas, for a total of 7 annular areas.

[0052] The radius ranges of the four first annular areas are: (Central proximal area), (Central proximal area), (middle section), (marginal distal area).

[0053] The radius ranges of the three second annular areas are: (Central proximal area), (middle section), (marginal distal area). Indicates the height or width of the colonoscopy image. The height is equal to the width.

[0054] In this preferred embodiment, exemplarily, the annular feature extractor includes a plurality of feature extraction networks corresponding to a plurality of annular regions. Each feature extraction network is used to extract the annular feature vector of its corresponding annular region. The feature extraction network includes a cascaded sparse convolution layer and an average pooling layer. For each annular region, all image regions (invalid regions) in the colonoscopy image except the image region (valid region) corresponding to the annular region are masked to obtain a mask image corresponding to each annular region. By processing the mask image corresponding to each annular region using the sparse convolution layer, it is possible to skip calculations for the invalid region and obtain only the image features corresponding to the annular region. The image features are then compressed using the average pooling layer to obtain the annular feature vector corresponding to the annular region.

[0055] In this preferred embodiment, see Figure 7,The encoder includes a feature splicing module, a position encoding module and a feature interaction module, which are connected in sequence.

[0056] The feature splicing module is used to splice multiple grid feature vectors generated by the square feature extractor and multiple ring feature vectors generated by the ring feature extractor into a mixed vector sequence.

[0057] In the above example, the feature concatenation module is used to concatenate the 36 grid feature vectors generated by the square feature extractor, i.e., the 36 grid token sequences. , and the 7 ring feature vectors generated by the ring feature extractor, that is, 7 ring token sequences Concatenate into mixed vector sequences , Indicates the The annular feature vector corresponding to the annular area, d = 768 is the feature dimension (ie the number of channels).

[0058] The position encoding module is used to inject a position encoding matrix into the mixed vector sequence. The position encoding matrix includes a position encoding vector corresponding to each grid eigenvector and a position encoding vector corresponding to each ring eigenvector. The position encoding vectors corresponding to the grid eigenvectors use standard grid coordinate encoding, while the position encoding vectors corresponding to the ring eigenvectors use ring polar coordinate encoding.

[0059] In the above example, the position encoding matrix is ​​represented as ,matrix The first 36 position encoding vectors are coded using standard grid encoding, and the last 7 position encoding vectors are coded using circular polar coordinates. , Indicates the normalized value of the annular radius of the annular area. The annular radius is the average of the minimum and maximum radii of the annular area.

[0060] The feature interaction module uses a multi-layer Transformer encoder structure. For example, a 12-layer Transformer encoder can be used. The multi-head attention mechanism in the Transformer encoder automatically establishes the mapping relationship between the grid feature vector, the ring feature vector and the classification label during classification model training. Therefore, the feature interaction module can obtain an accurate classification label feature vector according to the mapping relationship between the grid feature vector, the ring feature vector and the classification label obtained through training. In the above example, the classification label feature vector corresponding to the classification label [CLS] is .

[0061] In this preferred embodiment, the classifier is not limited to using a multi-layer perceptron (MLP) or a multi-layer cascade fully connected structure to achieve classification decisions. Preferably, the classifier uses a two-layer multi-layer perceptron (MLP) to obtain the classification result Expressed as:

[0062]

[0063] in, 、 Represent the weight matrix and bias matrix of the first fully connected layer in the two-layer multilayer perceptron (MLP), 、 Represent the weight matrix and bias matrix of the second fully connected layer in the two-layer multilayer perceptron (MLP), Represents the GELU activation function.

[0064] In an application scenario of this embodiment, the colonoscopic image detection method provided by this embodiment is used for classification of ulcerative colitis and Crohn's disease, and the classification results include confidence levels of the three categories of ulcerative colitis, Crohn's disease, and normal.

[0065] In another application scenario of this embodiment, the colonoscopic image detection method provided by this embodiment is used to classify the severity of ulcerative colitis, corresponding to pathological severity levels 0 to 3. The classification results include four levels of confidence: 0, 1, 2, and 3.

[0066] In another preferred embodiment of this embodiment, see Figure 5 , the ring feature extractor includes:

[0067] The preliminary feature extraction module extracts depth features of multiple annular areas to obtain multiple preliminary annular features.

[0068] The depth of field hierarchical processing module selects at least one expert model guided by the annular radius of the annular area, and uses the selected at least one expert model to process the annular preliminary features of the annular area to obtain an annular feature vector corresponding to the annular area.

[0069] In this preferred embodiment, preliminary annular features of the annular region are first extracted. To achieve feature extraction at different depths of field, at least one expert model is adaptively selected based on the annular radius of each annular region. The selected at least one expert model is used to process the preliminary annular features of the annular region to obtain the annular feature vector corresponding to the annular region, thus implementing a dynamic expert model selection mechanism. The expert model for the proximal region focuses on the villus microstructure, while the expert model for the distal region models the overall brightness distribution, thereby achieving differentiated extraction and fusion of high-resolution texture in the proximal region and global brightness features in the distal region. This improves the accuracy of intestinal disease classification and grading of the severity of ulcerative colitis.

[0070] In this preferred embodiment, the expert model preferably adopts, but is not limited to, a multi-layer perceptron (MLP) network, such as a two-layer MLP.

[0071] Further preferably, in order to more accurately extract the annular preliminary features of each annular area in the colonoscopy image, please refer to Figure 5 ,The preliminary feature extraction module includes:

[0072] The first branch processes the plurality of first annular regions respectively through a first sparse convolution processing module to obtain a plurality of first annular preliminary features;

[0073] The second branch processes the plurality of second annular regions respectively through a second sparse convolution processing module to obtain a plurality of second annular preliminary features;

[0074] The plurality of first annular preliminary features and the plurality of second annular preliminary features constitute a plurality of annular preliminary features.

[0075] In this preferred embodiment, see Figure 5 In the first branch, the first sparse convolution processing module includes a cascaded first sparse convolution layer and a first average pooling layer. First, a mask image corresponding to each first annular area is generated. The mask image corresponding to each first annular area is generated based on the colonoscopy image. In the mask image, only the image area corresponding to the first annular area is the valid area. All image areas except the image area corresponding to the first annular area are defined as invalid areas, and the invalid areas are masked. Then, the first sparse convolution layer is used to process the mask image corresponding to each first annular area respectively. The sparse convolution can ignore the mask part and only perform convolution processing on the valid part to obtain the convolution feature; then the first average pooling layer is used to compress the convolution feature of each first annular area to obtain the first annular preliminary feature corresponding to each first annular area. In the above example, the convolution kernel size of the first sparse convolution layer is 3×3, and the first average pooling layer is preferably a global average pooling layer, which obtains 4 256-dimensional first annular preliminary features.

[0076] In this preferred embodiment, see Figure 5In the second branch, the second sparse convolution processing module includes a cascaded second sparse convolution layer and a second average pooling layer. First, a mask image corresponding to each second annular region is generated. The mask image corresponding to each second annular region is generated based on the colonoscopy image. In this mask image, only the image area corresponding to the second annular region is the valid area. All image areas other than the image area corresponding to the second annular region are defined as invalid areas, and the invalid areas are masked. Then, the second sparse convolution layer is used to process the mask image corresponding to each second annular region separately. Sparse convolution can ignore the masked part and only perform convolution processing on the valid part to obtain convolution features; then, the second average pooling layer is used to compress the convolution features of each second annular region to obtain the second annular preliminary features corresponding to each second annular region. In the above example, the convolution kernel size of the second sparse convolution layer is 3×3, and the second average pooling layer is preferably a global average pooling layer, which obtains three 256-dimensional second annular preliminary features. The four first annular preliminary features and the three second annular preliminary features constitute seven annular preliminary features.

[0077] In this preferred embodiment, the depth of field features of each annular area (i.e., each annular band), such as the near-end texture and the far-end brightness gradient, are independently extracted through the first sparse convolutional layer and the second sparse convolutional layer to solve the problem of feature distortion caused by lens distortion.

[0078] Further preferably, for adaptive selection of expert models, see Figure 6 , the depth of field level processing module includes:

[0079] An expert model library, including multiple expert models, each of which corresponds to a ring radius of a different ring area;

[0080] A position coding unit, which adds a position coding vector to the preliminary annular feature of each annular area to obtain a feature position coding vector, wherein the position coding vector is obtained by coding the annular radius of the annular area;

[0081] The model selection unit calculates the product vector of the feature position encoding vector and the gate weight matrix of each annular region, and generates the expert weight vector corresponding to the annular region based on the product vector;

[0082] The expert fusion processing unit selects at least one expert model to process the feature position encoding vector of each annular area according to the expert weight vector corresponding to the annular area, and fuses the processing results of at least one expert model to obtain the annular feature vector corresponding to the annular area.

[0083] In this preferred embodiment, the expert model library defines Expert models, , In line with the number of annular regions, for example, ; Indicates the An expert model, which corresponds to the annular radius of an annular area.

[0084] In this preferred embodiment, the position encoding unit is actually a dynamic gating mechanism: for any preliminary annular feature of an annular region, such as for an annular region Preliminary ring characteristics , first calculate the annular area Positional encoding vector , For the annular area The annular radius The cosine position encoding, Finally, Splicing in preliminary ring features Then, we get the annular region The feature position encoding vector By changing the annular radius The cosine position encoding of the annular radius is used to inject the annular radius information, guiding the model to focus on the radial structural continuity of the intestinal lumen and enhancing the sensitivity to pathological features such as annular fold destruction.

[0085] In this preferred embodiment, in the step of calculating the product vector of the feature position encoding vector and the gate weight matrix of each annular region by the model selection unit, for the annular region , annular area The feature position encoding vector With the gating weight matrix The product vector is Preferably, the model selection unit generates the annular region according to the following formula: The corresponding expert weight vector :

[0086]

[0087] in, represents the gating weight matrix, , is a learnable matrix, specifically a linear layer; Represents a ring area The feature position encoding vector of Indicated in the product vector Keep the largest Item element, and not the largest The elements of the item are set to zero; Represents the normalization function. Greater than 1 and less than , preferably, That is, keep the product vector The values ​​of the two largest items, and the rest of the non-largest The elements of the item are set to zero, so that a new vector is obtained. After processing, the obtained expert weight vector To include A vector of elements, each of which is either 0 or 1. . Expert weight vector Each element corresponds to an expert model, indicating that the corresponding expert model has a good understanding of the annular area. The processing weight of the ring feature vector is 1, which means processing and 0 means no processing. The largest Item in The corresponding element in is 1, and the product vector Except this Any other item except The corresponding element in is 0.

[0088] In this preferred embodiment, the expert fusion processing unit selects at least one expert model to process the feature position encoding vector of each annular region according to the expert weight vector corresponding to the annular region, and fuses the processing results of at least one expert model to obtain the annular feature vector corresponding to the annular region. Specifically, the expert fusion processing unit selects at least one expert model to process the feature position encoding vector of the annular region according to the expert weight vector corresponding to the annular region, and fuses the processing results of at least one expert model to obtain the annular feature vector corresponding to the annular region. The processing process is:

[0089] ;

[0090] in, A learnable matrix, specifically a linear layer, is used to map the fused features of at least one expert model's processing results into a new feature space. This design aims to extract features at different depths of field by guiding the selection of the corresponding regional expert model using the annular radius. Represents a ring area The corresponding ring eigenvector. Represents a ring area The corresponding expert weight vector Middle and First Expert Model The corresponding element. Indicates the Expert Model For the annular area Preliminary ring characteristics The processing result.

[0091] Example 2

[0092] This embodiment provides a colonoscopy image detection device for implementing the colonoscopy image detection method provided in Example 1. Figure 8 As shown, in a preferred implementation of this embodiment, the device includes:

[0093] An image acquisition module, which acquires colonoscopy images;

[0094] The classification module inputs the colonoscopy images into the classification model to obtain the classification results.

[0095] The image acquisition module and the classification module correspond one-to-one to steps A1 and A2 in Example 1, and are not described again here.

[0096] Please refer to Figure 3 , the classification models include:

[0097] A square feature extractor is used to extract multiple grid feature vectors from colonoscopy images;

[0098] An annular template processing module divides the colonoscopy image into multiple annular regions based on an annular template, wherein the annular template includes multiple concentric rings, wherein valid rings and invalid rings are alternately distributed in the multiple concentric rings, and a concentric ring region is defined as an annular region;

[0099] A ring feature extractor, which obtains multiple ring feature vectors corresponding to multiple ring areas;

[0100] An encoder encodes the multiple grid feature vectors, the multiple ring feature vectors and the classification labels to obtain a classification label feature vector;

[0101] The classifier performs classification processing on the classification mark feature vector to obtain the classification result.

[0102] In this embodiment, the specific structure and working principle of the classification model have been described in detail in Example 1 and will not be repeated here.

[0103] Example 3

[0104] Related technologies that utilize a masked autoencoder (MAE)-based self-supervised learning method to train an encoder and decoder, enabling the encoder to accurately extract key discriminative features from colonoscopic images, only perform random mask reconstruction on a single colonoscopic image to learn local feature associations within that image, ignoring the high similarity between colonoscopic images across cases. This cross-case similarity is manifested in the highly restricted color distribution of colonoscopic images within a narrow red-yellow-white color gamut, primarily reflecting pathological conditions such as mucosal congestion and ulcers. This color monotony results in extremely high visual similarity between normal mucosal regions across different patients, while key discriminative features reside only in local microstructural variations (such as villus morphology and microvascular patterns). The single-sample learning model employed by these related technologies makes it difficult for the model to capture both common features across cases (such as the circular folds and narrow color gamut distribution of healthy mucosa) and subtle pathological differences (such as ulcer morphology and changes in villus structure), limiting the model's generalization ability for disease identification and classification.

[0105] Based on this, this embodiment provides a mask-based self-supervised learning method to solve the above problems based on the inherent characteristics of colonoscopy images and the high similarity across cases. The mask-based self-supervised learning method provided in this embodiment is used for pre-training of the square feature extractor, the ring feature extractor and the encoder in Example 1. The execution subject of the method includes but is not limited to at least one of the electronic devices such as the server and the terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the mask-based self-supervised learning method can be executed by software or hardware installed on the terminal device or the server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0106] In this embodiment, the mask-based self-supervised learning method includes:

[0107] Step B1: construct a sample pair set, wherein a sample pair is composed of two colonoscopy images randomly selected from the colonoscopy image set.

[0108] In one example, if Figure 9 As shown, two colonoscopy images are randomly selected from the unlabeled colonoscopy image set. and Form a sample pair.

[0109] Step B2: Generate a spatial decoupled view and a phase-reversed view for each sample pair using a checkerboard template, and use the phase-reversed view as the reconstruction target of the sample pair; and generate a ring-shaped fusion view including multiple ring-shaped areas for each sample pair using a ring-shaped template.

[0110] like Figure 9 The mask template part is pre-formed with two space-complementary checkerboard templates. For example, constructing space-complementary checkerboard templates and Checkerboard Template : 6×6 chessboard, starting with black squares, see Figure 9 The first checkerboard template from top to bottom in the middle mask template part. Checkerboard template : 6×6 chessboard, starting with white squares, see Figure 9 The second checkerboard template from top to bottom in the middle mask template. Checkerboard template Checkerboard template Pre-built ring template and ring template Phase-reversed annular templates forming a radial complementary pair ,template and See Figure 9 The third and fourth ring templates from top to bottom in the middle mask template part, template and This has been described in detail in Example 1 and will not be repeated here.

[0111] For example, three synthetic views are defined as:

[0112] Spatially decoupled view:

[0113] Phase-reversed view:

[0114] Annular fusion view:

[0115] in, Represents the pixel-by-pixel mask operation, and it can be seen that the three synthetic views retain the sample alignment at the same time and The local features and their spatial associations of the above template design are and It combines the grid features of natural images and uses the ring template and The unique annular topological features of the intestinal cavity are integrated to facilitate the extraction of grid feature vectors and annular feature vectors.

[0116] In this embodiment, the phase reversal view As the reconstruction target, the self-supervised learning model only needs to predict the missing small area based on the local features of neighboring blocks (such as texture, color continuity, etc.). The task complexity is low and the gradient update is more stable.

[0117] Step B3: construct a self-supervised learning model and train the self-supervised learning model using the spatial decoupling view, phase reversal view, and annular fusion view of the sample pair set.

[0118] Please refer to Figure 9 , self-supervised learning models include:

[0119] Square feature extractor for disentangling views from the space of sample pairs Extract multiple grid feature vectors; for example, Figure 9 As shown, 36 grid feature vectors of 16×16 pixel blocks are extracted. The specific structure of the square feature extractor has been described in detail in Example 1 and will not be repeated here.

[0120] The ring feature extractor obtains multiple ring feature vectors corresponding to multiple ring regions of the ring fusion view of the sample pair; for example, Figure 9 As shown, 7 annular feature vectors corresponding to 7 annular regions are extracted. The specific structure of the annular feature extractor has been described in detail in Example 1 and will not be repeated here.

[0121] The encoder performs encoding processing on multiple grid feature vectors and multiple ring feature vectors to obtain encoding features. The specific structure of the encoder has been described in detail in Example 1 and will not be repeated here. For example, Figure 9 As shown in Figure 2, during the training process of each sample pair, the encoded features output by the encoder include 43 tokens.

[0122] The decoder reconstructs the reconstruction target based on the encoded features after adding the learnable mask token to obtain the reconstructed image. For example, Figure 9 As shown, set the mask token for decoder learning ( ), the encoded features (43 tokens) output by the encoder are combined with the learnable mask token ( ) and then input the spliced ​​tokens into the decoder for decoding to obtain the reconstructed target. Specifically, the decoder reconstructs the original information by predicting the pixel values ​​of each mask token (i.e., mask block). The last layer of the decoder is a linear projection layer, whose output channels equal the number of pixels in each mask block. The decoder output is then reshaped to construct the reconstructed image.

[0123] In this embodiment, a 12-layer Transformer is used as the encoder, and its multi-head attention mechanism automatically builds and The pixel-level correspondence between the grid token and the ring token can be used to achieve cross-case association. In addition, the information carried by the ring token can be leaked to the grid token, reducing the reconstruction difficulty of the decoder.

[0124] In this embodiment, although it is required to reconstruct the complete phase reversal view However, there is knowledge transfer between samples within the sample pair, and the high similarity of colonoscopy images enables the model to use colonoscopy images and knowledge transfer based on the common features of The information of contains part of the information of the image block that needs to be reconstructed, that is, A portion of the feature vector will be leaked to the decoder to reconstruct the target image , therefore, the decoder in this embodiment can obtain a complete reconstructed image.

[0125] In this embodiment, step B3 iteratively trains the self-supervised learning model using the spatially decoupled views, phase-reversed views, and annular fusion views of the sample pair set. In each iterative training, the spatially decoupled views and annular fusion views of a sample pair are input to the square feature extractor and annular feature extractor of the self-supervised learning model. The decoder outputs a reconstructed image. A learning loss is calculated based on the reconstructed image and the reconstruction target. The network parameters of the self-supervised learning model are adjusted based on the learning loss, and the next iterative training continues until a training termination condition is met, such as when the number of iterative training iterations reaches a preset number or the learning loss falls below a preset loss threshold.

[0126] Learning loss Defined as:

[0127]

[0128] Pixel loss:

[0129]

[0130] Structural loss:

[0131]

[0132] in, Indicates the height of the colonoscopy image in pixels; Indicates the width of the colonoscopy image in pixels; Indicates the number of channels of the colonoscopy image; Represents the reconstructed image In position ,aisle Pixel value of Represents the phase-reversed view of the reconstructed target In position ,aisle Pixel value of Represents the reconstructed image The local mean of , reflecting the brightness characteristics; Represents the phase-reversed view of the reconstructed target The local mean of Represents the reconstructed image The local variance of Represents the phase-reversed view of the reconstructed target The local variance of Represents the reconstructed image and reconstruct the target phase-reversed view The covariance of , characterizing structural similarity; the first stability constant The value is 0.0001; the second stability constant The value is 0.0009; Indicates the dynamic range of pixel values. When the dynamic range of the pixel values ​​of the colonoscopy image is 0-255, . Represents the square operation of the L2 norm.

[0133] This embodiment has the following technical effects:

[0134] (1) Cross-case feature transfer: By utilizing the cross-case similarities of colonoscopy images (such as restricted color distribution and annular topological structure), the model is forced to learn the common features between cases (such as the annular continuity of healthy mucosa) and local differences (such as morphological destruction of lesion areas) through two-sample mask reconstruction, significantly improving the model's sensitivity to subtle pathological changes.

[0135] (2) Local-global feature collaboration: through pixel-level mask superposition (such as ), the model needs to simultaneously reconstruct local areas from different samples, forcing the encoder to establish semantic associations across images (such as the comparison between ulcer areas and healthy mucosa) and enhance the discriminability of feature expression.

[0136] (3) The decoder uses the radial information leaked by the ring mask (such as The complementary regions in the image are combined with cross-case knowledge from two samples to achieve more accurate image reconstruction. Compared to the random masking of MAE, this method uses a priori constraints on the ring structure to make the reconstruction process more consistent with the anatomical characteristics of the intestinal lumen, improving the feature quality of self-supervised pre-training.

[0137] Example 4

[0138] The present embodiment provides a classification model training method, the execution subject of the method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the classification model training method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0139] Please refer to Figure 10 , the classification model training methods include:

[0140] In the self-supervised pre-training stage, the steps of the mask-based self-supervised learning method provided in Example 3 are executed to obtain the pre-trained square feature extractor, ring feature extractor and encoder.

[0141] During the supervision fine-tuning phase, perform:

[0142] Step C1, such as Figure 10 As shown, a classification model is constructed using a ring template processing module, a classifier, and a square feature extractor, a ring feature extractor, and an encoder obtained from the end of the supervised pre-training phase;

[0143] Step C2, use the colonoscopy image classification sample set to train the classification model.

[0144] In this embodiment, the specific structures of the classification model, the ring template processing module and the classifier have been described in detail in Example 1 and will not be repeated here. The square feature extractor, the ring feature extractor and the encoder are obtained through self-supervised pre-training.

[0145] In this embodiment, the colonoscopic image classification sample set includes multiple colonoscopic images and the category label corresponding to each colonoscopic image. The category label definition varies depending on the application scenario of the classification model. When performing auxiliary diagnosis of ulcerative colitis and Crohn's disease, the category labels may include ulcerative colitis, Crohn's disease, and normal. When grading the pathological severity of ulcerative colitis, the category labels include four levels: 0, 1, 2, and 3. The input of the supervised fine-tuning stage is a single randomly sampled colonoscopic image. The square feature extractor directly processes the complete colonoscopic image (rather than the checkerboard splicing image in the self-supervised pre-training stage) to generate 36 grid tokens. The ring feature extractor also receives a complete single colonoscopic image, retains the generation logic of 7 ring tokens in Example 3, but removes the cross-sample fusion mechanism. The decoder of self-supervised learning is removed in the supervised fine-tuning stage.

[0146] In step C2, the classification model is trained using the colonoscopy image classification sample set. During the training process, the standard cross entropy loss function can be used to calculate the loss, and the network parameters of the classification model are adjusted according to the loss.

[0147] Example 5

[0148] This embodiment provides an electronic device, such as Figure 11 , which is a schematic diagram of the structure of an electronic device for executing the methods provided in Examples 1, 3, and 4 of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as the method programs provided in Examples 1, 3, and 4 of the present invention.

[0149] In some embodiments, the processor 10 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and circuits. It executes or executes programs or modules stored in the memory 11 (for example, executing the methods provided in Examples 1, 3, and 4 of the present invention), and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0150] The memory 11 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device. Furthermore, the memory 11 may include both an internal storage unit of the electronic device and an external storage device. The memory 11 can be used not only to store application software installed in the electronic device and various types of data, such as the code of the method program provided in Examples 1, 3, and 4 of the present invention, but also to temporarily store data that has been output or is about to be output.

[0151] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0152] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0153] Figure 11 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 11The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0154] For example, although not shown, the electronic device may further include a power source (e.g., a battery) to power various components. Preferably, the power source may be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charge management, discharge management, and power consumption management via the power management device. The power source may further include any of one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not further detailed here.

[0155] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0156] Furthermore, if the modules / units integrated into electronic devices are implemented as software functional units and sold or used as independent products, they can be stored on a computer-readable storage medium. Computer-readable storage media can be volatile or non-volatile. For example, computer-readable media may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, and read-only memory (ROM).

[0157] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A colonoscopy image detection method, characterized in that: include: Obtain colonoscopic images; Input the colonoscopy image into the classification model to obtain the classification results; Among them, the classification model includes: A square feature extractor is used to extract multiple grid feature vectors from colonoscopy images; The annular template processing module divides the colonoscopy image into a plurality of annular regions based on the annular template, wherein the annular template includes a plurality of concentric rings, wherein valid rings and invalid rings are alternately distributed in the plurality of concentric rings, and a concentric ring region is defined as an annular region; the annular template processing module performs: a masking operation on the colonoscopy image using the annular template to obtain a plurality of first annular regions; a masking operation on the colonoscopy image using a phase-reversed annular template that forms a radially complementary pair with the annular template to obtain a plurality of second annular regions; the plurality of first annular regions and the plurality of second annular regions constitute a plurality of annular regions; A ring feature extractor, which obtains multiple ring feature vectors corresponding to multiple ring areas; An encoder encodes the multiple grid feature vectors, the multiple ring feature vectors and the classification labels to obtain a classification label feature vector; The classifier performs classification processing on the classification mark feature vector to obtain the classification result; Wherein, the annular feature extractor comprises: A preliminary feature extraction module extracts depth features of multiple annular areas to obtain multiple preliminary annular features; a depth of field hierarchical processing module, which selects at least one expert model guided by the annular radius of the annular region, and uses the selected at least one expert model to process preliminary annular features of the annular region to obtain an annular feature vector corresponding to the annular region; wherein the expert model adopts a multi-layer perceptron network; Wherein, the preliminary feature extraction module includes: The first branch processes the plurality of first annular regions respectively through the first sparse convolution processing module to obtain a plurality of first annular preliminary features; generates a mask image corresponding to each first annular region; processes the mask image corresponding to each first annular region respectively through the first sparse convolution layer to obtain convolution features; compresses the convolution features of each first annular region through the first average pooling layer to obtain the first annular preliminary features corresponding to each first annular region; The second branch processes the plurality of second annular regions separately through a second sparse convolution processing module to obtain a plurality of second annular preliminary features; generates a mask image corresponding to each second annular region; processes the mask image corresponding to each second annular region separately through a second sparse convolution layer to obtain a convolution feature; and compresses the convolution feature of each second annular region through a second average pooling layer to obtain a second annular preliminary feature corresponding to each second annular region; The plurality of first annular preliminary features and the plurality of second annular preliminary features constitute a plurality of annular preliminary features.

2. The colonoscopy image detection method according to claim 1, wherein: The depth of field level processing module includes: An expert model library, including multiple expert models, each of which corresponds to a ring radius of a different ring area; A position coding unit, which adds a position coding vector to the preliminary annular feature of each annular area to obtain a feature position coding vector, wherein the position coding vector is obtained by coding the annular radius of the annular area; The model selection unit calculates the product vector of the feature position encoding vector and the gate weight matrix of each annular region, and generates the expert weight vector corresponding to the annular region based on the product vector; The expert fusion processing unit selects at least one expert model to process the feature position encoding vector of each annular area according to the expert weight vector corresponding to the annular area, and fuses the processing results of at least one expert model to obtain the annular feature vector corresponding to the annular area.

3. The colonoscopy image detection method according to claim 2, wherein: The model selection unit generates the annular region according to the following formula The corresponding expert weight vector : in, represents the gating weight matrix, Represents a ring area The feature position encoding vector of Indicated in the product vector Keep the largest Item element, and not the largest The elements of the item are set to zero; Represents the normalization function.

4. A colonoscopy image detection device, characterized in that: A method for detecting colonoscopy images according to any one of claims 1 to 3, comprising: An image acquisition module, which acquires colonoscopy images; The classification module inputs the colonoscopy images into the classification model to obtain the classification results; Among them, the classification model includes: A square feature extractor is used to extract multiple grid feature vectors from colonoscopy images; The annular template processing module divides the colonoscopy image into a plurality of annular regions based on the annular template, wherein the annular template includes a plurality of concentric rings, wherein valid rings and invalid rings are alternately distributed in the plurality of concentric rings, and a concentric ring region is defined as an annular region; the annular template processing module performs: a masking operation on the colonoscopy image using the annular template to obtain a plurality of first annular regions; a masking operation on the colonoscopy image using a phase-reversed annular template that forms a radially complementary pair with the annular template to obtain a plurality of second annular regions; the plurality of first annular regions and the plurality of second annular regions constitute a plurality of annular regions; A ring feature extractor, which obtains multiple ring feature vectors corresponding to multiple ring areas; An encoder encodes the multiple grid feature vectors, the multiple ring feature vectors and the classification labels to obtain a classification label feature vector; The classifier performs classification processing on the classification mark feature vector to obtain the classification result; Wherein, the annular feature extractor comprises: A preliminary feature extraction module extracts depth features of multiple annular areas to obtain multiple preliminary annular features; a depth of field hierarchical processing module, which selects at least one expert model guided by the annular radius of the annular region, and uses the selected at least one expert model to process preliminary annular features of the annular region to obtain an annular feature vector corresponding to the annular region; wherein the expert model adopts a multi-layer perceptron network; Wherein, the preliminary feature extraction module includes: The first branch processes the plurality of first annular regions respectively through the first sparse convolution processing module to obtain a plurality of first annular preliminary features; generates a mask image corresponding to each first annular region; processes the mask image corresponding to each first annular region respectively through the first sparse convolution layer to obtain convolution features; compresses the convolution features of each first annular region through the first average pooling layer to obtain the first annular preliminary features corresponding to each first annular region; The second branch processes the plurality of second annular regions separately through a second sparse convolution processing module to obtain a plurality of second annular preliminary features; generates a mask image corresponding to each second annular region; processes the mask image corresponding to each second annular region separately through a second sparse convolution layer to obtain a convolution feature; and compresses the convolution feature of each second annular region through a second average pooling layer to obtain a second annular preliminary feature corresponding to each second annular region; The plurality of first annular preliminary features and the plurality of second annular preliminary features constitute a plurality of annular preliminary features.

5. A mask-based self-supervised learning method, characterized in that include: Construct a sample pair set, where a sample pair is composed of two colonoscopy images randomly sampled from the colonoscopy image set; A checkerboard template is used to generate a spatially decoupled view and a phase-reversed view for each sample pair, with the phase-reversed view being the reconstruction target for the sample pair; an annular template is used to generate an annular fused view comprising a plurality of annular regions for each sample pair; a masking operation is performed on the annular fused view using the annular template to obtain a plurality of first annular regions; a masking operation is performed on the annular fused view using a phase-reversed annular template that forms a radially complementary pair with the annular template to obtain a plurality of second annular regions; the plurality of first annular regions and the plurality of second annular regions constitute a plurality of annular regions; A self-supervised learning model is constructed and trained using the spatial decoupling view, phase reversal view, and ring fusion view of the sample pair set. The self-supervised learning model includes: A square feature extractor for extracting multiple grid feature vectors from spatially decoupled views of sample pairs; a ring feature extractor, which obtains a plurality of ring feature vectors corresponding to a plurality of ring regions of the ring fusion view of the sample pair; An encoder, encoding the multiple grid feature vectors and the multiple ring feature vectors to obtain encoding features; The decoder reconstructs the reconstruction target based on the encoded features after adding the learnable mask token to obtain the reconstructed image; Wherein, the annular feature extractor comprises: A preliminary feature extraction module extracts depth features of multiple annular areas to obtain multiple preliminary annular features; a depth of field hierarchical processing module, which selects at least one expert model guided by the annular radius of the annular region, and uses the selected at least one expert model to process preliminary annular features of the annular region to obtain an annular feature vector corresponding to the annular region; wherein the expert model adopts a multi-layer perceptron network; Wherein, the preliminary feature extraction module includes: The first branch processes the plurality of first annular regions respectively through the first sparse convolution processing module to obtain a plurality of first annular preliminary features; generates a mask image corresponding to each first annular region; processes the mask image corresponding to each first annular region respectively through the first sparse convolution layer to obtain convolution features; compresses the convolution features of each first annular region through the first average pooling layer to obtain the first annular preliminary features corresponding to each first annular region; The second branch processes the plurality of second annular regions separately through a second sparse convolution processing module to obtain a plurality of second annular preliminary features; generates a mask image corresponding to each second annular region; processes the mask image corresponding to each second annular region separately through a second sparse convolution layer to obtain a convolution feature; and compresses the convolution feature of each second annular region through a second average pooling layer to obtain a second annular preliminary feature corresponding to each second annular region; The plurality of first annular preliminary features and the plurality of second annular preliminary features constitute a plurality of annular preliminary features.

6. A classification model training method, characterized in that: include: In the self-supervised pre-training stage, the steps of the mask-based self-supervised learning method according to claim 5 are performed; During the supervision fine-tuning phase, perform: A classification model is constructed using the ring template processing module, the classifier, and the square feature extractor, ring feature extractor, and encoder obtained from the supervised pre-training phase. Use the colonoscopy image classification sample set to train the classification model; The annular template processing module performs: masking operations on the colonoscopy image classification samples using the annular template to obtain a plurality of first annular regions; masking operations on the colonoscopy image classification samples using a phase-reversed annular template that forms a radially complementary pair with the annular template to obtain a plurality of second annular regions; the plurality of first annular regions and the plurality of second annular regions form a plurality of annular regions; Wherein, the annular feature extractor comprises: A preliminary feature extraction module extracts depth features of multiple annular areas to obtain multiple preliminary annular features; a depth of field hierarchical processing module, which selects at least one expert model guided by the annular radius of the annular region, and uses the selected at least one expert model to process preliminary annular features of the annular region to obtain an annular feature vector corresponding to the annular region; wherein the expert model adopts a multi-layer perceptron network; Wherein, the preliminary feature extraction module includes: The first branch processes the plurality of first annular regions respectively through the first sparse convolution processing module to obtain a plurality of first annular preliminary features; generates a mask image corresponding to each first annular region; processes the mask image corresponding to each first annular region respectively through the first sparse convolution layer to obtain convolution features; compresses the convolution features of each first annular region through the first average pooling layer to obtain the first annular preliminary features corresponding to each first annular region; The second branch processes the plurality of second annular regions separately through a second sparse convolution processing module to obtain a plurality of second annular preliminary features; generates a mask image corresponding to each second annular region; processes the mask image corresponding to each second annular region separately through a second sparse convolution layer to obtain a convolution feature; and compresses the convolution feature of each second annular region through a second average pooling layer to obtain a second annular preliminary feature corresponding to each second annular region; The plurality of first annular preliminary features and the plurality of second annular preliminary features constitute a plurality of annular preliminary features.

7. An electronic device, characterized in that: The electronic device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-3, 5, and 6.

Images