Semi-supervised learning-based colonoscope lesion recognition training method and electronic equipment

Abstract

The invention discloses a colonoscope lesion recognition training method based on semi-supervised learning and electronic equipment. The method comprises the steps that paired label image data pairs and an image data set only containing colonoscope lesion are acquired; designing a lightweight lesion recognition network, wherein the lightweight lesion recognition network comprises a teacher network model and a student network model; performing network training parameter initialization on the lightweight lesion recognition network; inputting the label image data pair and the image data set only containing colonoscope lesion into a lightweight lesion identification network for training; when the joint loss function value of the student network model is smaller than the empirical value, the lightweight lesion recognition network training is completed; and identifying the input colonoscope lesion image by using the trained student network model to obtain an identification result. The adaptive capacity of the training model to complex lesion features is improved, the generalization of the training model is enhanced, and therefore efficient and reliable automatic lesion detection is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a training method and electronic equipment for colonoscopic lesion recognition based on semi-supervised learning. Background Art

[0002] The high cost of labeling medical image datasets is primarily due to the need for professional medical personnel to accurately annotate each frame of a colonoscopy image or video. This is not only time-consuming and labor-intensive, but also requires a high level of professional knowledge. Faced with the complex and diverse morphologies of different lesions, such as polyps, ulcers, and hyperplasia, labeling often requires physicians to make judgments based on their experience, and labeling consistency can be affected by individual differences. Furthermore, with the rapid growth in the scale of imaging data, traditional manual labeling methods are unable to meet the needs of large-scale medical image analysis, further limiting the construction of high-quality medical image datasets.

[0003] While existing few-shot recognition methods can achieve certain results when data is limited, they lack adaptability when faced with the high-dimensional, complex features and diverse lesion morphologies found in medical images. Specifically, few-shot recognition methods typically rely on general domain features or pre-trained models. These models perform well when processing general image datasets, but often underperform in the medical field due to significant differences in data distribution and feature expression. Furthermore, when existing methods migrate across domains, the models lack sensitivity to new category features, which can easily lead to a decrease in recognition accuracy. Therefore, few-shot learning techniques for medical image data urgently need to be further optimized to improve their adaptability to complex lesion features and enhance model generalization, thereby achieving efficient and reliable automated lesion detection. Summary of the Invention

[0004] The purpose of the present invention is to provide a training method and electronic equipment for colonoscopic lesion recognition based on semi-supervised learning, improve the adaptability of the training model to complex lesion characteristics, enhance the generalization of the training model, and thus achieve efficient and reliable automated lesion detection.

[0005] In order to achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] A training method for colonoscopic lesion recognition based on semi-supervised learning, comprising:

[0007] Step S1, obtaining paired labeled image data pairs and an image dataset containing only colonoscopic lesions;

[0008] Step S2: designing a lightweight lesion recognition network, wherein the lightweight lesion recognition network includes a teacher network model and a student network model;

[0009] Step S3: Initializing network training parameters of the lightweight lesion recognition network;

[0010] Step S4: inputting the labeled image data pairs and the image data set containing only colonoscopic lesions into the lightweight lesion recognition network for training;

[0011] Step S5: When the joint loss function value of the student network model is less than the empirical value, the lightweight lesion recognition network training is completed;

[0012] Step S6: using the trained student network model to identify the input colonoscopic lesion image to obtain a recognition result.

[0013] Optionally, step S1 includes:

[0014] Step S1.1, preparing a paired labeled image dataset and a dataset containing only colonoscopic lesion images; the amount of data in the dataset containing only colonoscopic lesion images is greater than the amount of data in the paired labeled image dataset;

[0015] Step S1.2: In each training iteration, batches of data are randomly sampled from the paired labeled image dataset and the colonoscopic lesion-only image dataset.

[0016] Optionally, step S4 includes:

[0017] Step S4.1: After weak enhancement processing is performed on the paired labeled image data, the data is input into a student network model for training to obtain a first prediction result, and a supervision loss is calculated based on the first prediction result and the labels in the paired labeled image data;

[0018] Step S4.2: After weak enhancement processing is performed on the image dataset containing only colonoscopic lesions, the image dataset is input into the teacher network model for training to obtain a second prediction result, and a pseudo label is calculated based on the second prediction result;

[0019] Step S4.3: After performing strong enhancement processing on the image dataset containing only colonoscopic lesions, the dataset is input into the student network model for training to obtain a third prediction result, and the consistency loss is calculated based on the third prediction result;

[0020] Step S4.4: weighted combination of supervision loss and consistency loss to obtain the joint loss function value;

[0021] Step S4.5, update the parameters of the student network model by combining the loss function value, and update the parameters of the teacher network model from the student network model by exponential moving average;

[0022] Step S5: Repeat steps S4.1 to S4.5 for multiple iterative training until the value of the joint loss function is less than the empirical value, then stop the training of the teacher network model and the student network model, and generate a trained student network model.

[0023] Optionally, step S2 includes: the teacher network model and the student network model have the same structure; the structure of the teacher network model includes: an input layer, which is used to normalize the input image data to obtain a normalized image tensor as an initial input of the teacher network model;

[0024] The initial convolution layer performs a convolution operation on the normalized image tensor to capture the local area information of the image and outputs a preliminary feature map;

[0025] The RepMCBlock network module stacking layer is used to perform in-depth processing on the preliminary feature map and output a high-level semantic feature map;

[0026] The middle convolutional layer is used to perform multi-level feature fusion processing on the low-level features, high-level features and multi-scale features in the high-level semantic feature map to obtain the initial fused feature map;

[0027] A residual connection layer, which performs feature fusion on the preliminary feature map and the initially fused feature map through a skip connection to obtain a fused feature map;

[0028] FFT, used to convert the spatial domain features in the fused feature map into frequency domain features;

[0029] The RepMCBlock network module is used to perform feature extraction operations on the frequency domain features to obtain a frequency domain feature map.

[0030] IFFT is used to convert the frequency domain features in the frequency domain feature map into spatial features to obtain the reconstructed feature map;

[0031] Output layers, including the final convolutional or fully connected layers,

[0032] The accumulated reconstructed feature maps and normalized image tensors are weighted through fully connected layers or convolutional layers to generate prediction results.

[0033] Optionally, the local area information of the image includes: basic visual features in a normalized image tensor extracted through a 3x3 convolutional layer, and the basic visual features include at least edge, texture and corner feature information of the image.

[0034] Optionally, the RepMCBlock network module stacking layer includes multiple serially connected RepMCBlock network modules, each RepMCBlock network module includes multi-scale point convolution to extract features of different scales;

[0035] During the training phase, each RepMCBlock network module uses a multi-scale point convolution structure to capture multi-scale contextual information in the time and frequency domains. After local feature extraction, it integrates global contextual information through global average pooling and multi-head self-attention modules.

[0036] In the inference phase, the complex multi-branch structure in the training phase is equivalent to a lightweight single-branch convolutional network, while retaining multi-scale context information;

[0037] The self-attention mechanism and multi-layer perceptron are introduced to mine the global dependencies between features; the input features and output results are added through residual connections to obtain and output high-level semantic feature maps.

[0038] Optionally, the step S4.2 further includes:

[0039] Screening high confidence samples: If max(P teacher )≥τ, retain the pseudo label of the sample, otherwise discard it; where τ is the threshold, P teacher is the second prediction result.

[0040] Optionally, the supervision loss L sup The calculation is done using the following formula:

[0041] L sup =CrossEntropy(P student1 ,Y l )

[0042] Among them, CrossEntropy is the cross entropy loss function; Y l represents the label in the paired label image data; P student1 Indicates the first prediction result;

[0043] The pseudo labels calculated in step S4.2 include hard labels and soft labels. The hard labels are The soft tag is P teacher ;

[0044] The consistency loss L in step S4.3 consistency The calculation is done using the following formula:

[0045]

[0046] Where Π(·) is an indicator function used to filter samples that meet the confidence condition; N represents the total number of current training samples; i represents the iteration index of the current sample in the batch; CrossEntropy represents the cross entropy loss function; represents the third prediction result of the i-th iteration training; represents the second prediction result of the i-th iteration training;

[0047] The joint loss function value in step S4.4 is calculated using the following formula:

[0048] L total =L sup +λ·L consistency

[0049] Among them, λ represents the consistency loss weight; in the process of back propagation, through the joint loss function L total Update the network training parameters of the student network model. The teacher network model does not participate in the gradient calculation.

[0050] Optionally, the network training parameters of the teacher network model are updated from the student network model via exponential moving average:

[0051]

[0052] Among them, α is the smoothing coefficient, which controls the parameter update speed; represents the network training parameters of the teacher network model at time t, Last moment t-1 The network training parameters of the teacher network model; Shows the network training parameters of the student network model at time t.

[0053] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described above is implemented.

[0054] The present invention has the following technical effects:

[0055] The present invention provides a training method for colonoscopic lesion recognition using semi-supervised learning. This method utilizes a small amount of labeled images and a large amount of unlabeled image data to collaboratively train a student network model and a teacher network model. The teacher network model updates its parameters using an exponential moving average and generates pseudo-labels for weakly augmented unlabeled data to guide the learning of the student network model. The student network model, combined with training on strongly augmented data, achieves improved lesion recognition efficiency and generalization capabilities, enhancing the training model's adaptability to complex lesion characteristics and generalization, thereby achieving efficient and reliable automated lesion detection.

[0056] The present invention also provides a lightweight network structure for semi-supervised learning, which uses a multi-scale point convolution structure to capture multi-scale contextual information in the time domain and frequency domain. After local feature extraction, the global contextual information is integrated through global average pooling and multi-head self-attention modules to improve the training convergence speed and reasoning performance of the student network and the teacher network. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic flow chart of a training method for colonoscopic lesion recognition based on semi-supervised learning provided by one embodiment of the present invention;

[0058] Figure 2 A schematic diagram of a training process using a teacher network model and a student network model according to an embodiment of the present invention;

[0059] Figure 3 A diagram showing the structure of a teacher network model and a student network model provided in one embodiment of the present invention;

[0060] Figure 4 A network module diagram of each RepMCBlock in the training phase and the inference phase provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following is a further detailed description of a training method and electronic device for colonoscopic lesion recognition based on semi-supervised learning proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, please refer to the drawings. It should be noted that the structure, proportion, size, etc. illustrated in the drawings of this specification are only used to match the content disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the implementation conditions of the present invention, so they have no technical significance. Any modification of the structure, change in the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0062] Colorectal cancer is one of the most common and deadly cancers in both men and women worldwide. Abnormalities such as polyps and ulcers are often early precursors and are frequently discovered during colonoscopy screening in people over 50 years old. Colonoscopy is considered the gold standard for the detection and early diagnosis of such abnormalities, often requiring real-time biopsies, which are crucial for improving the survival rate of colorectal cancer patients. Automated detection of lesions during colonoscopy not only reduces the risk of missed lesions, but also reduces the workload of gastroenterologists and improves screening efficiency. Regular colonoscopy and imaging follow-up should be performed during the recovery phase to closely monitor changes in the condition and ensure early detection of possible recurrence or metastasis.

[0063] In view of this, the present invention improves screening efficiency and realizes efficient and reliable automated lesion detection by providing a training method for colonoscopic lesion recognition based on semi-supervised learning.

[0064] like Figure 1 As shown, this embodiment provides a training method for colonoscopic lesion recognition based on semi-supervised learning, including:

[0065] Step S1, obtaining paired labeled image data pairs and an image dataset containing only colonoscopic lesions;

[0066] Step S2: designing a lightweight lesion recognition network, wherein the lightweight lesion recognition network includes a teacher network model and a student network model;

[0067] Step S3: Initializing network training parameters of the lightweight lesion recognition network;

[0068] Step S4: inputting the labeled image data pairs and the image data set containing only colonoscopic lesions into the lightweight lesion recognition network for training;

[0069] Step S5: When the joint loss function value of the student network model is less than the empirical value, the lightweight lesion recognition network training is completed;

[0070] Step S6: using the trained student network model to identify the input colonoscopic lesion image to obtain a recognition result.

[0071] This embodiment provides a training method for colonoscopic lesion recognition using semi-supervised learning. This method utilizes a small amount of labeled images and a large amount of unlabeled image data to collaboratively train a student network model and a teacher network model. The teacher network model updates its parameters using an exponential moving average and generates pseudo-labels for weakly augmented unlabeled data to guide the learning of the student network model. The student network model, combined with training on strongly augmented data, achieves improved lesion recognition efficiency and generalization capabilities, enhancing the training model's adaptability to complex lesion characteristics and generalization, thereby achieving efficient and reliable automated lesion detection.

[0072] In this embodiment, step S1 includes: step S1.1, preparing a small amount of paired labeled image datasets and a large amount of colonoscopic lesion image datasets (unlabeled image datasets).

[0073] Step S1.2, data sampling: In each training iteration, batches of data are randomly sampled from the paired label image dataset and the colonoscopy lesion image dataset. The paired label data batches are: (X l ,Y l ) Unlabeled image dataset batch: X u .

[0074] The network structures of the teacher network model and the student network model are as follows: Figure 3 As shown, the teacher network model and the student network model provided in this embodiment have the same network structure, and the weights in the initial stage of training are randomly initialized.

[0075] The network structure of the teacher network model and the student network model mainly includes the following 7 parts:

[0076] The input layer normalizes the input image data to obtain a normalized image tensor as the initial input to the network. For example, the input image data is normalized to [0, 1] or standardized to a mean of 0 and a variance of 1 before serving as the initial input to the network. The normalized image tensor has a shape of [Batch Size, Channels, Height, Width].

[0077] The initial convolutional layer 100 is, for example, a 3x3 convolutional layer that extracts preliminary feature information. It performs a convolution operation on the normalized image tensor to capture local image region information and outputs a preliminary feature map. The shape is [Batch Size, Feature Channels, Height, Width], where Feature Channels is the number of output channels of the convolutional layer.

[0078] The RepMCBlock network module stacking layer 101 is used to perform in-depth processing on the preliminary feature map output by the initial convolutional layer 100 and output a high-level semantic feature map.

[0079] In this embodiment or some other embodiments, as Figure 4As shown, the RepMCBlock network module stacking layer includes multiple series-connected RepMCBlock network modules, through which the features are deeply processed; each RepMCBlock network module contains multi-scale point convolution (for example, 3x3, 5x5, 7x7) to extract features of different scales; and combines global pooling and normalization operations to enhance feature expression capabilities.

[0080] In particular, during the training phase, each RepMCBlock network module uses a multi-scale point convolution structure to capture multi-scale contextual information in the time and frequency domains. After local feature extraction, it integrates global contextual information through global average pooling and multi-head self-attention modules.

[0081] During the inference phase, the complex multi-branch structure of the training phase is equivalent to a lightweight single-branch convolutional network, which greatly reduces network computing latency and memory access requirements while retaining the ability to express multi-scale contextual information.

[0082] The self-attention mechanism (Multi-head Self Attention, MSA) and Multilayer Perceptron (Multilayer Perceptron, MLP) are introduced to explore the global dependency relationship between features; the input features and output results are added through residual connections (which can promote gradient flow and improve training stability) to obtain and output high-level semantic feature maps. It can be understood that the high-level semantic feature maps, shapes and input features Figure 1 The lesion is similar to the original image, but contains richer semantic information (such as the edge, shape, texture, etc. of the lesion).

[0083] This embodiment uses a multi-scale point convolution structure to capture multi-scale contextual information in the time domain and frequency domain. After local feature extraction, it integrates global contextual information through global average pooling and multi-head self-attention modules to improve the training convergence speed and inference performance of the student network model and the teacher network model.

[0084] The middle convolution layer 102 (e.g., 3x3 convolution layer) is used to perform multi-level feature fusion processing on the low-level features (e.g., edge, texture), high-level features (e.g., (shape, structure) shape, structure) and multi-scale features in the high-level semantic feature map to obtain the initial fused feature map. It can be understood that the initial fused feature map, shape and input feature map Figure 1 But the feature expression ability is stronger.

[0085] The residual connection layer 103 performs feature fusion on the preliminary feature map and the initially fused feature map through a skip connection to obtain a fused feature map.

[0086] That is, in this embodiment, the input features are directly added to the output of the intermediate convolutional layer through the jump connection to achieve feature fusion. This operation retains the low-level feature information and can enhance the expressiveness of the features. The fused feature map has a shape similar to the input feature. Figure 1 To.

[0087] An FFT (Fast Fourier Transform) 104 is used to convert the spatial domain features in the fused feature map into frequency domain features and extract frequency domain features (such as global information of texture and edge).

[0088] The RepMCBlock network module 105 is used to perform a feature extraction operation on the frequency domain features to obtain a frequency domain feature map.

[0089] It can be understood that the RepMCBlock network module 105 expands the capabilities of traditional convolution through frequency domain operations, making it easier to capture global structures or periodic patterns (such as repeated textures and large-scale shapes in images) because frequency domain operations can directly adjust components of different frequencies; and each RepMCBlock network module in the RepMCBlock network module stacking layer 101 focuses on detailed features such as local textures and edges (local correlation in the spatial domain).

[0090] IFFT (Inverse Fast Fourier Transform) 106 is used to convert the frequency domain features in the frequency domain feature map into spatial features to obtain a reconstructed feature map. It can be understood that the reconstructed feature map has a shape similar to the input feature map. Figure 1 It is consistent with the original image, but contains dual information in frequency domain and spatial domain.

[0091] Output layer, including the final convolutional layer 107 or fully connected layer 108,

[0092] The accumulated reconstructed feature maps and normalized image tensors are weighted and processed through the fully connected layer 108 or the final convolutional layer 107 to generate prediction results (such as lesion category probability or lesion location information). The final output of the network has a shape of [Batch Size, Num Classes] (classification task) or [Batch Size, Num Boxes, 4 + Num Classes] (detection task).

[0093] Optionally, the local area information of the image includes: basic visual features in a normalized image tensor extracted through a 3x3 convolutional layer, and the basic visual features include at least edge, texture and corner feature information of the image.

[0094] The step S3 includes: after designing the model structures of the student network and the teacher network in step S2, the network parameters need to be initialized. Specifically: the student network model is initialized to random network training parameters θ student ; Teacher network model: initialized to the same structure as the student network model, but the network training parameters θ teacher are initialized independently; the network training parameters of the teacher network model are updated by the exponential moving average (EMA) of the student network model.

[0095] The execution process of network training is as follows Figure 2 As shown, in each training iteration, the process described in step S4 is executed. Step S4 includes:

[0096] Step S4.1: After weak enhancement processing is performed on the paired labeled image data, the data is input into the student network model for training to obtain a first prediction result, and the supervision loss is calculated based on the first prediction result and the label in the paired labeled image data.

[0097] It can be understood that step S4.1 is a supervised learning (paired labeled image data samples, student network model) process:

[0098] Input: batches of paired labeled image data (X l ,Y l )

[0099] Execution steps:

[0100] a. Image data X l Apply weak enhancement (such as random flipping, resizing) to obtain weakly enhanced image data

[0101] b. Input into the student network model to obtain the first prediction result P student1 : Among them, f student Represents the output value after the input value passes through the student network model;

[0102] c. Calculate the supervision loss L sup :

[0103] L sup =CrossEntropy(P student1 ,Y l )

[0104] Among them, CrossEntropy is the cross entropy loss function, Y l Represents the labels in a batch of paired labeled image data.

[0105] Step S4.2: After weak enhancement processing is performed on the image dataset containing only colonoscopic lesions, the image dataset is input into the teacher network model for training to obtain a second prediction result, and a pseudo label is calculated based on the second prediction result.

[0106] Specifically, step S4.2 is the process of pseudo-label generation (unlabeled data, teacher network model);

[0107] Input: Unlabeled image data sample X u ;

[0108] Execution steps:

[0109] a. For unlabeled image data sample X u Apply weak enhancement (such as random flipping, resizing) to obtain weakly enhanced unlabeled image data samples

[0110] b. Unlabeled image data samples after weak enhancement Input the teacher network model and get the second prediction result Among them, f teacher Represents the output value after the input value passes through the teacher network model;

[0111] c. Generate pseudo labels

[0112] Calculating pseudo labels (as a hard label) or keep P teacher (as a soft tag).

[0113] Screening high confidence samples: If max(P teacher )≥τ (τ is the threshold, for example, τ=0.8), the pseudo label of the sample is retained, otherwise it is discarded.

[0114] Step S4.3: After performing strong enhancement processing on the image dataset containing only colonoscopic lesions, the dataset is input into the student network model for training to obtain a third prediction result, and the consistency loss is calculated based on the third prediction result.

[0115] Specifically, consistency regularization (unlabeled image data sample X u , student network model)

[0116] Input: The unlabeled image data sample X is input to the teacher network u Belong to the same batch.

[0117] step:

[0118] a. For unlabeled image data sample X u Apply strong enhancement (such as random rotation, shearing, color jitter and Gaussian noise) to obtain strongly enhanced unlabeled image data samples

[0119] b. The strongly enhanced unlabeled image data samples Input into the student network model to obtain the third prediction result

[0120] c. Calculate the consistency loss L consistency :

[0121] Only for high confidence levels, the loss is strictly not calculated (that is, when the maximum probability value (that is, confidence level) of the student network model's prediction result for the enhanced sample exceeds the threshold τ, the sample is included in the calculation of the consistency loss):

[0122]

[0123] Where ∏(·) is an indicator function that filters samples that meet the confidence criteria. N represents the total number of current training samples; i represents the iteration index of the current sample in the batch; represents the third prediction result of the i-th iteration training; Represents the second prediction result of the i-th iteration training. Generate consistency loss L consistency , and then perform gradient backpropagation based on the loss value and model structure of the student network model.

[0124] Step S4.4: Weightedly combine the supervision loss and consistency loss to obtain the joint loss function value.

[0125] The joint loss function value in step S4.4 is calculated using the following formula:

[0126] L total =L sup +λ·L consistency

[0127] Where λ represents the consistency loss weight.

[0128] In the process of back propagation, through the joint loss function L total Update the network training parameters of the student network model. The teacher network model does not participate in the gradient calculation.

[0129] The backpropagation process uses the chain rule to calculate the gradient of the joint loss function with respect to the student network model parameters, thereby updating the student network model's training parameters. Gradient calculation: During backpropagation, the gradient starts from the joint loss function and propagates backward through each layer of the student network model (using the chain rule), updating the student network model's training parameters. The teacher network model does not participate in the gradient calculation: the teacher network model does not update its weight parameters.

[0130] Step S4.5: Update the parameters of the student network model through the joint loss function value, and update the parameters of the teacher network model from the student network model through exponential moving average.

[0131] Specifically, the network training parameters of the teacher network model are updated from the student network model through exponential moving average:

[0132]

[0133] Among them, α is the smoothing coefficient, which controls the parameter update speed; represents the network training parameters of the teacher network model at time t, Last moment t-1 The network training parameters of the teacher network model; Shows the network training parameters of the student network model at time t.

[0134] Step S5: Repeat steps S4.1 to S4.5 for multiple iterative training until the value of the joint loss function is less than the empirical value, then stop the training of the teacher network model and the student network model, and generate a trained student network model.

[0135] The step S6 includes: using the trained student network model to identify the input colonoscopy lesion image, and outputting the predicted result after calculation, that is, the lesion type.

[0136] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described above is implemented.

[0137] In summary, the present invention provides a training method for colonoscopic lesion recognition using semi-supervised learning, which utilizes a small amount of labeled images and a large amount of unlabeled image data to perform collaborative training through a student network model and a teacher network model. The teacher network model updates its parameters through exponential moving average and generates pseudo-labels for weakly enhanced unlabeled data to guide the learning of the student network model. The student network model is combined with strong enhanced data training to achieve high efficiency and improved generalization of lesion recognition, improve the training model's adaptability to complex lesion characteristics, and enhance the generalization of the training model, thereby achieving efficient and reliable automated lesion detection.

[0138] The present invention also provides a lightweight network structure for semi-supervised learning, which uses a multi-scale point convolution structure to capture multi-scale contextual information in the time domain and frequency domain. After local feature extraction, the global contextual information is integrated through global average pooling and multi-head self-attention modules to improve the training convergence speed and reasoning performance of the student network and the teacher network.

[0139] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0140] It should be noted that the devices and methods disclosed in the embodiments of this document may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the devices, methods, and computer program products according to the various embodiments of this document. In this regard, each box in the flowchart or block diagram may represent a module, program, or portion of code, wherein the module, program segment, or portion of code contains one or more executable instructions for implementing a specified logical function, and the module, program segment, or portion of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

[0141] In addition, the functional modules in the various embodiments of this document may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0142] Although the present invention has been described in detail through the above preferred embodiments, it should be understood that the above description is not intended to limit the present invention. After reading the above description, various modifications and substitutions of the present invention will become apparent to those skilled in the art. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A training method for colonoscopic lesion recognition based on semi-supervised learning, characterized in that: include: Step S1, obtaining paired labeled image data pairs and an image dataset containing only colonoscopic lesions; Step S2: designing a lightweight lesion recognition network, wherein the lightweight lesion recognition network includes a teacher network model and a student network model; Step S3: Initializing network training parameters of the lightweight lesion recognition network; Step S4: inputting the labeled image data pairs and the image data set containing only colonoscopic lesions into the lightweight lesion recognition network for training; Step S5: When the joint loss function value of the student network model is less than the empirical value, the lightweight lesion recognition network training is completed; Step S6: using the trained student network model to identify the input colonoscopic lesion image to obtain a recognition result.

2. The colonoscopic lesion recognition training method based on semi-supervised learning as described in claim 1 is characterized in that: The step S1 comprises: Step S1.1, preparing a paired labeled image dataset and a dataset containing only colonoscopic lesion images; the amount of data in the dataset containing only colonoscopic lesion images is greater than the amount of data in the paired labeled image dataset; Step S1.2: In each training iteration, batches of data are randomly sampled from the paired labeled image dataset and the colonoscopic lesion-only image dataset.

3. The colonoscopic lesion recognition training method based on semi-supervised learning as described in claim 2 is characterized in that: The step S4 comprises: Step S4.1: After weak enhancement processing is performed on the paired labeled image data, the data is input into a student network model for training to obtain a first prediction result, and a supervision loss is calculated based on the first prediction result and the labels in the paired labeled image data; Step S4.2: After weak enhancement processing is performed on the image dataset containing only colonoscopic lesions, the image dataset is input into the teacher network model for training to obtain a second prediction result, and a pseudo label is calculated based on the second prediction result; Step S4.3: After performing strong enhancement processing on the image dataset containing only colonoscopic lesions, the dataset is input into the student network model for training to obtain a third prediction result, and the consistency loss is calculated based on the third prediction result; Step S4.4: weighted combination of supervision loss and consistency loss to obtain the joint loss function value; Step S4.5, update the parameters of the student network model by combining the loss function value, and update the parameters of the teacher network model from the student network model by exponential moving average; Step S5: Repeat steps S4.1 to S4.5 for multiple iterative training until the value of the joint loss function is less than the empirical value, then stop the training of the teacher network model and the student network model, and generate a trained student network model.

4. The colonoscopic lesion recognition training method based on semi-supervised learning as described in claim 3 is characterized in that: The step S2 includes: the teacher network model and the student network model have the same structure; The structure of the teacher network model includes: an input layer, which is used to normalize the input image data to obtain a normalized image tensor as the initial input of the teacher network model; The initial convolution layer performs a convolution operation on the normalized image tensor to capture the local area information of the image and outputs a preliminary feature map; The RepMCBlock network module stacking layer is used to perform in-depth processing on the preliminary feature map and output a high-level semantic feature map; The middle convolutional layer is used to perform multi-level feature fusion processing on the low-level features, high-level features and multi-scale features in the high-level semantic feature map to obtain the initial fused feature map; A residual connection layer, which performs feature fusion on the preliminary feature map and the initially fused feature map through a skip connection to obtain a fused feature map; FFT, used to convert the spatial domain features in the fused feature map into frequency domain features; The RepMCBlock network module is used to perform feature extraction operations on the frequency domain features to obtain a frequency domain feature map. IFFT is used to convert the frequency domain features in the frequency domain feature map into spatial features to obtain the reconstructed feature map; Output layers, including the final convolutional or fully connected layers, The accumulated reconstructed feature maps and normalized image tensors are weighted through fully connected layers or convolutional layers to generate prediction results.

5. The colonoscopic lesion recognition training method based on semi-supervised learning as described in claim 4 is characterized in that: The local area information of the image includes: basic visual features in the normalized image tensor extracted through a 3x3 convolution layer, and the basic visual features include at least edge, texture and corner feature information of the image.

6. The colonoscopic lesion recognition training method based on semi-supervised learning as described in claim 4 is characterized in that: The RepMCBlock network module stacking layer includes multiple serially connected RepMCBlock network modules, each of which includes multi-scale point convolution to extract features of different scales; During the training phase, each RepMCBlock network module uses a multi-scale point convolution structure to capture multi-scale contextual information in the time and frequency domains. After local feature extraction, it integrates global contextual information through global average pooling and multi-head self-attention modules. In the inference phase, the complex multi-branch structure in the training phase is equivalent to a lightweight single-branch convolutional network, while retaining multi-scale context information; The self-attention mechanism and multi-layer perceptron are introduced to mine the global dependencies between features; the input features and output results are added through residual connections to obtain and output high-level semantic feature maps.

7. The colonoscopic lesion recognition training method based on semi-supervised learning as claimed in claim 4, characterized in that: The step S4.2 further includes: Screening high confidence samples: If max(P teacher )≥τ, retain the pseudo label of the sample, otherwise discard it; where τ is the threshold, P teacher is the second prediction result.

8. The colonoscopic lesion recognition training method based on semi-supervised learning as claimed in claim 7, characterized in that: The supervision loss L sup The calculation is done using the following formula: L sup =CrossEntropy(P student1 ,Y l ) Among them, CrossEntropy is the cross entropy loss function; Y l represents the label in the paired label image data; P student1 Indicates the first prediction result; The pseudo labels calculated in step S4.2 include hard labels and soft labels. The hard labels are The soft tag is P teacher ; The consistency loss L in step S4.3 consistency The calculation is done using the following formula: Where Π(·) is an indicator function used to filter samples that meet the confidence condition; N represents the total number of current training samples; i represents the iteration index of the current sample in the batch; CrossEntropy represents the cross entropy loss function; represents the third prediction result of the i-th iteration training; represents the second prediction result of the i-th iteration training; The joint loss function value in step S4.4 is calculated using the following formula: L total =L sup +λ·L consistency Among them, λ represents the consistency loss weight; in the process of back propagation, through the joint loss function L total Update the network training parameters of the student network model. The teacher network model does not participate in the gradient calculation.

9. The colonoscopic lesion recognition training method based on semi-supervised learning as claimed in claim 8, characterized in that: The network training parameters of the teacher network model are updated from the student network model through exponential moving average: Among them, α is the smoothing coefficient, which controls the parameter update speed; represents the network training parameters of the teacher network model at time t, Last moment t-1 The network training parameters of the teacher network model; Shows the network training parameters of the student network model at time t.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 9 is implemented.

Images