Gastrointestinal image reconstruction method, device and electronic device based on semi-supervised learning

Abstract

The present invention discloses a method, device and electronic device for gastrointestinal image reconstruction based on semi-supervised learning, the method comprising: obtaining image data related to gastrointestinal images, dividing the image data into labeled first image data and unlabeled second image data, filtering the unlabeled second image data according to a linear filter; conducting supervised training on a deep learning model according to the labeled first image data; conducting unsupervised training on a deep learning model according to the filtered unlabeled second image data, the deep learning model is used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and unsupervised training is performed according to the difference between the data of the second reconstructed image and the unlabeled second image data; obtaining the gastrointestinal image to be processed, and inputting it into the trained deep learning model to determine the reconstructed image. The present invention can identify gastrointestinal images more efficiently and accurately through reconstruction, which is convenient for subsequent judgment.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a gastrointestinal image reconstruction method, device and electronic equipment based on semi-supervised learning. Background Art

[0002] In modern medicine, imaging examination is an important means of clinical diagnosis, which can help doctors accurately identify and analyze the patient's condition.

[0003] However, some existing medical-related imaging images have poor image quality, resulting in low accuracy of image recognition results by doctors or models. Summary of the invention

[0004] The present invention provides a gastrointestinal image reconstruction method, device and electronic device based on semi-supervised learning, which can perform supervised and unsupervised training on a deep learning model through a large amount of training data, and then use the trained deep learning model to reconstruct the image, so as to perform more accurate recognition based on the reconstructed image.

[0005] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0006] In a first aspect, the present application provides a gastrointestinal image reconstruction method based on semi-supervised learning, the method comprising: acquiring image data related to gastrointestinal images, dividing the image data into labeled first image data and unlabeled second image data, and filtering the unlabeled second image data according to a linear filter; performing supervised training on a deep learning model according to the labeled first image data, the deep learning model being used to reconstruct the labeled first image data to obtain a first reconstructed image, and performing supervised training based on the difference between the data of the first reconstructed image and the labeled first image data; performing unsupervised training on the deep learning model according to the filtered unlabeled second image data, the deep learning model being used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and performing unsupervised training based on the difference between the data of the second reconstructed image and the unlabeled second image data; acquiring the gastrointestinal image to be processed, and inputting it into the trained deep learning model to determine the reconstructed image.

[0007] Preferably, the step of filtering the unlabeled second image data with the linear filter includes: obtaining the pixel value and pixel coordinates of each pixel point of the unlabeled second image data; filtering the pixel value of the pixel point based on the pixel value, pixel coordinates, and pixel values ​​of the pixel point's adjacent pixels to determine the filtered unlabeled second image data.

[0008] Preferably, the filtered unlabeled second image data is determined according to the following formula:

[0009] ,

[0010] Wherein, L represents the total number of pixels in the filtered unlabeled second image data, The pixel coordinates are The pixel value of the pixel at the position, represents the Laplace function, is the direction parameter of the Laplace function, is with The coordinates of the pixels adjacent to the position are The pixel values ​​of the adjacent pixels, Q represents the linear filter function, represents the linear filter factor parameter, , is the smoothing factor parameter and satisfies , represents the unlabeled second image data after filtering.

[0011] Preferably, the deep learning model includes an encoder and a decoder, the encoder includes multiple encoding layers, the encoding layers include convolution layers and pooling layers, and the decoder includes multiple decoding layers; the deep learning model is unsupervisedly trained based on the filtered unlabeled second image data, including: extracting image features of the filtered unlabeled second image data through the encoding layer of the encoder to obtain output features of the encoder; processing the output features of the encoder through the decoding layer of the decoder to obtain output features of the decoder; reconstructing the image based on the output features of the decoder to obtain a second reconstructed image; determining the adjustment of parameters of the deep learning model based on the difference between the data of the second reconstructed image and the unlabeled second image data and the difference between the output features of the encoder and the output features of the decoder, so as to perform unsupervised training on the deep learning model.

[0012] Preferably, the image features of the filtered unlabeled second image data are extracted through the encoding layer of the encoder, and the output features of the encoder are obtained according to the following formula:

[0013] ,

[0014] in, is the output feature of the encoder at layer l, is the encoder convolution kernel of layer l, is the encoder bias term of the lth layer, represents the average pooling operation of the pooling layer, Represents the convolution operation of the convolutional layer.

[0015] Preferably, the output feature of the encoder is processed by the decoding layer of the decoder, and the output feature of the decoder is determined according to the following formula:

[0016] ,

[0017] in is the output feature of the encoder at layer l, is the output feature of the decoder at layer l, represents the upsampling operation, represents the transposed convolution operation, represents the decoder convolution kernel of layer l, is the decoder bias term of layer l.

[0018] Preferably, an optimizer is further included between the encoder and the decoder, and the optimizer is used to adjust the encoder parameters of the next layer according to the output characteristics of the encoder and the output characteristics of the decoder to determine the output characteristics of the adjusted encoder;

[0019] The optimizer determines the output characteristics of the adjusted encoder according to the following formula:

[0020] ,

[0021] in, represents the optimizer feedback factor of the lth layer, represents the gradient operation, is the output feature of the encoder at layer l, is the output feature of the decoder at layer l, Represents the characteristic residual function, which is used to calculate and The characteristic differences between represents the output features of the encoder of layer l+1 after passing through the optimizer, Represents the learning rate.

[0022] Preferably, the difference between the data of the second reconstructed image and the unlabeled second image data is determined according to the following formula:

[0023] ,

[0024] Wherein, M represents the difference between the data of the second reconstructed image and the unlabeled second image data, that is, the loss function, is the filtered unlabeled second image data, is the data of the second reconstructed image, N represents the total number of features of the encoder and decoder, represents the convolution operation of the convolutional layer, is the dynamic adjustment factor of the loss function, is the output feature of the decoder at layer l, is the output feature of the encoder at layer l, express norm; among them, , is the output feature of the decoder at layer l, represents the tanh function, Represents a transposed convolution operation.

[0025] In a second aspect, the present application provides a gastrointestinal image reconstruction device based on semi-supervised learning, the device comprising: an image data acquisition module, used to acquire image data related to gastrointestinal images, divide the image data into labeled first image data and unlabeled second image data, and filter the unlabeled second image data according to a linear filter; a first model training module, used to perform supervised training on a deep learning model based on the labeled first image data, the deep learning model is used to reconstruct the labeled first image data to obtain a first reconstructed image, and supervised training is performed based on the difference between the data of the first reconstructed image and the labeled first image data; a second model training module, used to perform unsupervised training on the deep learning model based on the filtered unlabeled second image data, the deep learning model is used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and unsupervised training is performed based on the difference between the data of the second reconstructed image and the unlabeled second image data; a reconstructed image generation module, used to acquire the gastrointestinal image to be processed, and input it into the trained deep learning model to determine the reconstructed image.

[0026] In a third aspect, the present application provides an electronic device comprising: a memory and at least one processor; the memory is used to store computer-executable instructions; the at least one processor is used to execute the computer-executable instructions stored in the memory, so that the at least one processor executes the method described in the first aspect.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0028] The present application can be applied in the scenario of reconstructing image data related to gastrointestinal images. The present solution can establish a deep learning model, which is used to extract features from the input image and reconstruct it based on the extracted features to obtain a higher quality reconstructed image so as to identify it based on the reconstructed image. Specifically, the present solution can first add annotations to the image data to obtain annotated image data and unannotated image data, and then selectively filter the unannotated image data, thereby reducing the amount of filtered data, wherein the filtering can be based on the pixel value of the pixel point in the image, the position of the pixel point in the image, and the pixel value of the adjacent pixel point, so as to be more suitable for the processing of image data related to gastrointestinal images. Afterwards, the deep learning model can be supervised trained using the annotated image data, and the deep learning model can be unsupervised trained using the unannotated image data, so as to obtain a trained deep learning model. After the deep learning model is trained, the gastrointestinal image to be processed can be reconstructed based on the deep learning model, so as to obtain a higher quality reconstructed image, so that the doctor or recognition model can identify it based on the reconstructed image. This solution uses some labeled image data and a large amount of unlabeled image data to train the deep learning model, thereby improving the quality of images reconstructed by the deep learning model, and can more accurately identify gastrointestinal images based on the reconstructed images. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0030] Figure 1 It is a flowchart of a gastrointestinal image reconstruction method based on semi-supervised learning according to an embodiment of the present application;

[0031] Figure 2 It is a structural schematic diagram of a gastrointestinal image reconstruction device based on semi-supervised learning according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In modern medicine, imaging examination is an important means of clinical diagnosis, which can help doctors accurately identify and analyze the patient's condition. However, the acquisition and processing of image data often face many challenges, especially in the case of insufficient labeled data, how to improve the image quality and reconstruction effect has become an urgent problem to be solved. The gastrointestinal image reconstruction method of the present invention utilizes semi-supervised learning to improve the reconstruction quality of gastrointestinal images in order to serve medical diagnosis. The method first preprocesses the gastrointestinal image data to ensure the quality and consistency of the input data. By constructing a deep learning network structure of an encoder and a decoder, image features can be effectively extracted and images can be reconstructed. In addition, the present invention adopts a semi-supervised loss function in the design of the loss function to ensure that the model can find effective feature associations in complex image data, and at the same time, combined with the strategy of dynamically adjusting the unsupervised loss weight, the learning performance of the model under a variety of data distributions is further guaranteed. The method proposed by the present invention can realize real-time reconstruction of gastrointestinal images, providing data support for doctors to make decisions in clinical applications.

[0034] In modern medicine, medical imaging is an important tool for diagnosis and treatment, and its development and application have gradually become an indispensable part of clinical medicine. The early diagnosis, monitoring and treatment of gastrointestinal diseases often rely on high-quality medical images, such as CT (computed tomography), MRI (magnetic resonance imaging) and endoscopic images. These images not only help doctors understand the patient's internal condition, but also help formulate appropriate treatment plans. However, with the advancement of data acquisition technology, the amount and complexity of image data have increased significantly, making image processing and reconstruction face many challenges. Traditional medical image reconstruction methods mostly rely on labeled sample data for deep learning, which requires a large amount of labeled data for model learning. Although with the development of machine learning technology, researchers have made certain progress in image segmentation, feature extraction and image reconstruction, but the cost of obtaining labeled sample data is high, and in many cases, professional medical knowledge and experience require a lot of manpower for data labeling. Therefore, in the practical application of image processing, the scarcity of labeled data has become an important factor limiting the improvement of model performance. In view of this, unsupervised learning, as a new learning model, has begun to attract more and more researchers. Unsupervised learning does not rely on labeled data. It can build models by analyzing a large amount of unlabeled data, aiming to explore the potential structure and patterns of the data itself. However, unsupervised learning often fails to provide accurate results when processing complex medical images because the lack of guidance for specific tasks leads to unsatisfactory feature extraction and reconstruction effects of the model during training. Therefore, the application of traditional unsupervised learning methods in medical image reconstruction is still limited. In order to solve the problem of insufficient labeled data, semi-supervised learning has become a research field that has received widespread attention in recent years. Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data, and uses the knowledge provided in the labeled data to guide the model to mine more information from the unlabeled data. This method not only effectively improves learning efficiency, but also improves the generalization ability of the model. In the field of medical image processing, semi-supervised learning has shown its potential to effectively improve the quality and accuracy of image reconstruction. Specifically, the advantage of semi-supervised learning is that it can provide a preliminary framework for the model with the help of a small number of labeled samples, while using unlabeled data to enrich the model's understanding of data structure and sample characteristics. This learning method is particularly suitable for the field of medical imaging, because in clinical practice, although a large amount of medical imaging data can often be collected, due to the lack of appropriate labels, these data are difficult to be directly used for model training. Therefore, the use of semi-supervised learning can not only improve the efficiency of data utilization, but also enhance the model's adaptability to complex imaging data in the absence of annotations. In this context, the present invention aims to optimize the reconstruction process of gastrointestinal medical images by integrating existing labeled data with abundant unlabeled data.This method applies semi-supervised learning technology for the first time to effectively solve the task of gastrointestinal image processing. The structural design introduces a deep learning network of encoder and decoder, with the aim of effectively capturing the characteristic information of the image data and reconstructing the original image.

[0035] Traditional supervised learning methods rely mainly on a large amount of labeled data for model training. Although this method allows the model to learn accurately based on the existing data annotation, the challenge it faces is that the acquisition of labeled data usually requires professional knowledge and a lot of manpower and material resources, which is not only time-consuming but also costly. In the field of medical imaging, it is particularly difficult to obtain a large amount of labeled medical imaging data, and training can often only rely on a small number of labeled samples, which greatly limits the performance and applicability of the model. Unlike traditional supervised learning, unsupervised learning methods can process unlabeled data and learn by exploring the potential structure and features of the data. However, unsupervised learning often lacks guidance for specific tasks, so its learning effect may be limited when processing tasks such as medical imaging that require high precision and high reliability. This makes the application of unsupervised learning in medical image reconstruction have certain limitations. Because traditional methods are often affected by noise and complex structures when processing medical images, the quality of reconstructed images is poor, which affects doctors' diagnosis. At the same time, traditional methods have certain limitations when capturing complex features in high-dimensional medical images, and cannot fully reflect the data structure and feature information of the images. In response to the above problems, semi-supervised learning technology came into being and has shown broad application prospects in the field of medical image processing. Semi-supervised learning can effectively improve the learning ability of the model by combining a small amount of labeled data with a large amount of unlabeled data. This technology uses the rich information contained in the labeled data to provide basic guidance for the model, while at the same time using unlabeled data to assist the model in exploring the potential features of the data and forming a more comprehensive feature representation. All of this makes semi-supervised learning particularly suitable for use in medical image processing, because in actual clinical environments, there is an urgent need for fast and efficient image reconstruction methods.

[0036] The technical solution of the present invention: Data preprocessing: improve image quality, reduce noise, enhance features, and provide high-quality input data for subsequent model training. Image reconstruction model design: design a deep learning model that can reconstruct the original image from the features. Loss function design: design a loss function that can balance supervised learning and unsupervised learning, and dynamically adjust weights to optimize model performance. Image reconstruction analysis: improve the application effect of reconstructed images in actual clinical diagnosis and the convenience of doctors in the image interpretation process. Image reconstruction analysis can design an image reconstruction model, and the image reconstruction model can be used to reconstruct gastrointestinal images. By comparing the reconstructed image with the original image, it can ensure that it can provide high-quality reconstructed images, which can assist doctors in making more accurate and efficient diagnoses.

[0037] This scheme uses semi-supervised learning image reconstruction technology to more accurately capture subtle features in gastrointestinal images, thereby improving the accuracy of disease recognition. The application of semi-supervised learning enables the model to use unlabeled data to further enhance its ability to recognize disease features. The automated processing of the model reduces the manual operation of doctors in the image interpretation process, thereby shortening the diagnosis time. The dynamically adjusted loss function ensures that the model can maintain a high learning efficiency at different training stages and accelerates the model training process. The reconstructed image has higher quality and more obvious features, making it more intuitive and convenient for doctors to interpret images. Through model-assisted diagnosis, doctors can identify diseases faster and reduce the possibility of misdiagnosis and missed diagnosis. This scheme proposes a semi-supervised learning gastrointestinal image reconstruction method, which can accurately reconstruct the original image from the extracted features to ensure the quality and accuracy of the reconstructed image. This scheme proposes an efficient image denoising method to ensure the quality of the input data and the clarity of the features. And innovative loss functions, including a combination of supervised loss and unsupervised loss, and a dynamic weight adjustment mechanism.

[0038] Specifically, the present application embodiment provides a gastrointestinal image reconstruction method based on semi-supervised learning, such as Figure 1 As shown, the method includes:

[0039] Step 102, acquiring image data related to gastrointestinal imaging, dividing the image data into first image data with annotations and second image data without annotations, and filtering the second image data without annotations according to a linear filter;

[0040] Step 104: performing supervised training on a deep learning model based on the labeled first image data, wherein the deep learning model is used to reconstruct the labeled first image data to obtain a first reconstructed image, and to perform supervised training based on a difference between data of the first reconstructed image and the labeled first image data;

[0041] Step 106: performing unsupervised training on a deep learning model based on the filtered unlabeled second image data, wherein the deep learning model is used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and performing unsupervised training based on a difference between data of the second reconstructed image and the unlabeled second image data;

[0042] Step 108: Obtain the gastrointestinal image to be processed and input it into the trained deep learning model to determine the reconstructed image.

[0043] The present application can divide the image data into gastrointestinal related areas and gastrointestinal unrelated areas or gastrointestinal central areas and gastrointestinal marginal areas, so as to divide the image data into labeled or unlabeled areas. For example, the area where the gastrointestinal tract may have hidden dangers (for example, the gastrointestinal high-frequency lesion area based on historical data or the gastrointestinal disease existence area based on a specific condition) can be set to be unlabeled, and the area outside the hidden danger area (the gastrointestinal low-frequency lesion area or the gastrointestinal marginal area) can be set to be labeled, but the present invention is not limited to this.

[0044] The present application can be applied in the scenario of reconstructing image data related to gastrointestinal images. The present solution can establish a deep learning model, which is used to extract features from the input image and reconstruct it based on the extracted features to obtain a higher quality reconstructed image so as to identify it based on the reconstructed image. Specifically, the present solution can first add annotations to the image data to obtain annotated image data and unannotated image data, and then selectively filter the unannotated image data, thereby reducing the amount of filtered data, wherein the filtering can be based on the pixel value of the pixel point in the image, the position of the pixel point in the image, and the pixel value of the adjacent pixel point, so as to be more suitable for the processing of image data related to gastrointestinal images. Afterwards, the deep learning model can be supervised trained using the annotated image data, and the deep learning model can be unsupervised trained using the unannotated image data, so as to obtain a trained deep learning model. After the deep learning model is trained, the gastrointestinal image to be processed can be reconstructed based on the deep learning model, so as to obtain a higher quality reconstructed image, so that the doctor or recognition model can identify it based on the reconstructed image. This solution uses some labeled image data and a large amount of unlabeled image data to train the deep learning model, thereby improving the quality of images reconstructed by the deep learning model, and can more accurately identify gastrointestinal images based on the reconstructed images.

[0045] In the present solution, the process of filtering the pixel values ​​of the pixel points in the image data can be filtered according to the position of the pixel point in the image and the pixel values ​​of one or more adjacent pixel points. Specifically, as an optional embodiment, the step of filtering the unlabeled second image data by the linear filter includes: obtaining the pixel value of each pixel point of the unlabeled second image data and the pixel coordinates of the pixel point; filtering the pixel value of the pixel point according to the pixel value, pixel coordinates, and pixel values ​​of the adjacent pixel points of the pixel point to determine the filtered unlabeled second image data. Specifically, as an optional embodiment, the filtered unlabeled second image data is determined according to the following formula:

[0046] ,

[0047] Wherein, L represents the total number of pixels in the filtered unlabeled second image data, The pixel coordinates are The pixel value of the pixel at the position, represents the Laplace function, is the direction parameter of the Laplace function, is with The coordinates of the pixels adjacent to the position are The pixel values ​​of the adjacent pixels, Q represents the linear filter function, represents the linear filter factor parameter, , is the smoothing factor parameter and satisfies , represents the unlabeled second image data after filtering.

[0048] This solution can pre-process the input gastrointestinal imaging data to improve the efficiency and effect of subsequent model training. The present invention designs a linear filter with an adjustable smoothing factor to filter the noise signal of gastrointestinal imaging data. The advantage is that it smoothes the image, reduces the random noise in the image while retaining the basic structure of the image. While smoothing the noise, the filter has a small degree of blurring on the edge of the image, which helps to maintain the important features of the image. The degree of smoothing can be controlled by adjusting the smoothing factor. By changing the smoothing factor, the image can be filtered to different degrees to reduce artifacts caused by noise. At the same time, nonlinear distortion will not be introduced, ensuring the stability of the processing process.

[0049] In addition, this solution can divide the image data into gastrointestinal related areas and gastrointestinal unrelated areas, and then determine the positional relationship between the pixel point and the gastrointestinal related area (including the edge and center, etc.) for the gastrointestinal related area, and then adopt the corresponding smoothing factor parameters for filtering.

[0050] The deep learning model in this solution may include an encoder and a decoder, the encoder is used to extract features from the image data to form a feature map (or output features), and the decoder is used to decode the features extracted by the encoder, thereby determining the output features for image reconstruction. Specifically, as an optional embodiment, the deep learning model includes an encoder and a decoder, the encoder includes multiple encoding layers, the encoding layers include convolution layers and pooling layers, and the decoder includes multiple decoding layers; the unsupervised training of the deep learning model based on the filtered unlabeled second image data includes: extracting the image features of the filtered unlabeled second image data through the encoding layer of the encoder to obtain the output features of the encoder; processing the output features of the encoder through the decoding layer of the decoder to obtain the output features of the decoder; reconstructing the image based on the output features of the decoder to obtain a second reconstructed image; determining the adjustment of the parameters of the deep learning model based on the difference between the data of the second reconstructed image and the unlabeled second image data and the difference between the output features of the encoder and the output features of the decoder to perform unsupervised training on the deep learning model.

[0051] This solution adopts a deep learning model with an encoder-decoder structure. The encoder part is used to extract the features of gastrointestinal images, and the decoder part reconstructs the extracted features into the original image. Encoder: The encoder is composed of a stack of convolutional layers and pooling layers, and its purpose is to extract the feature representation of the input image. Each layer outputs a set of feature maps. Specifically, as an optional embodiment, the process of determining the feature map (or output feature) by the encoder is determined according to the following formula:

[0052] ,

[0053] in, is the output feature of the encoder at layer l, is the encoder convolution kernel of layer l, is the encoder bias term of the lth layer, represents the average pooling operation of the pooling layer, Represents the convolution operation of the convolutional layer.

[0054] Decoder: The decoder gradually upsamples and refines the output feature map of the encoder, and finally reconstructs the original image or the desired output. Specifically, as an optional embodiment, the process of determining the output feature of the decoder is determined according to the following formula:

[0055] ,

[0056] in is the output feature of the encoder at layer l, is the output feature of the decoder at layer l, represents the upsampling operation, represents the transposed convolution operation, represents the decoder convolution kernel of layer l, Decoder bias term for layer l.

[0057] This solution also sets an optimizer between the encoder and the decoder, and the optimizer can adjust the output features of the encoder's next layer according to the output features of the encoder and the decoder's previous layer, thereby improving the accuracy of the model. Specifically, as an optional embodiment, the encoder and the decoder also include an optimizer, and the optimizer is used to adjust the encoder parameters of the next layer of the target layer according to the output features of the encoder of the target layer and the output features of the decoder to determine the output features of the adjusted encoder.

[0058] The optimizer determines the output characteristics of the adjusted encoder according to the following formula:

[0059] ,

[0060] in, represents the optimizer feedback factor of the lth layer, represents the gradient operation, is the output feature of the encoder at layer l, is the output feature of the decoder at layer l, Represents the characteristic residual function, which is used to calculate and The characteristic differences between represents the output features of the encoder of layer l+1 after passing through the optimizer, Represents the learning rate.

[0061] The design of the loss function: including the combination of supervised loss and unsupervised loss. This solution adopts the strategy of dynamically adjusting the unsupervised loss weights to optimize the weights of each part of the loss function according to the actual performance in the training phase, ensuring that the model can maintain good learning performance under different data characteristics. Specifically, as an optional embodiment, the loss function is determined according to the following formula:

[0062] ,

[0063] Where M is the difference between the data of the second reconstructed image and the unlabeled second image data, that is, the loss function, is the filtered unlabeled second image data, is the data of the second reconstructed image, N represents the total number of features of the encoder and decoder, represents the convolution operation of the convolutional layer, is the dynamic adjustment factor of the loss function, is the output feature of the decoder at layer l, is the output feature of the encoder at layer l, express norm, where , is the output feature of the decoder at layer l, represents the tanh function, Represents a transposed convolution operation.

[0064] Some solutions adjust the model based on the difference between the image before and after reconstruction. This solution convolves the images before and after reconstruction to obtain features, and fine-tunes the model based on the difference between the features and the difference between the output features of the encoder layer and the decoder layer, which can better optimize the model parameters.

[0065] On the basis of the above embodiments, the embodiments of the present application further provide a gastrointestinal image reconstruction device based on semi-supervised learning, such as Figure 2 As shown, the device comprises:

[0066] An image data acquisition module 202 is used to acquire image data related to gastrointestinal imaging, divide the image data into first image data with annotations and second image data without annotations, and filter the second image data without annotations according to a linear filter;

[0067] A model first training module 204, configured to perform supervised training on a deep learning model based on the labeled first image data, wherein the deep learning model is configured to reconstruct the labeled first image data to obtain a first reconstructed image, so as to perform supervised training based on a difference between data of the first reconstructed image and the labeled first image data;

[0068] A model second training module 206, configured to perform unsupervised training on a deep learning model based on the filtered unlabeled second image data, wherein the deep learning model is used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and perform unsupervised training based on a difference between data of the second reconstructed image and the unlabeled second image data;

[0069] The reconstructed image generation module 208 is used to obtain the gastrointestinal image to be processed and input it into the trained deep learning model to determine the reconstructed image.

[0070] The implementation of the embodiment of the present application is similar to the implementation of the above-mentioned method embodiment. The specific implementation can refer to the specific implementation of the above-mentioned method embodiment, which will not be repeated here.

[0071] The present application can be applied in the scenario of reconstructing image data related to gastrointestinal images. The present solution can establish a deep learning model, which is used to extract features from the input image and reconstruct it based on the extracted features to obtain a higher quality reconstructed image so as to identify it based on the reconstructed image. Specifically, the present solution can first add annotations to the image data to obtain annotated image data and unannotated image data, and then selectively filter the unannotated image data, thereby reducing the amount of filtered data, wherein the filtering can be based on the pixel value of the pixel point in the image, the position of the pixel point in the image, and the pixel value of the adjacent pixel point, so as to be more suitable for the processing of image data related to gastrointestinal images. Afterwards, the deep learning model can be supervised trained using the annotated image data, and the deep learning model can be unsupervised trained using the unannotated image data, so as to obtain a trained deep learning model. After the deep learning model is trained, the gastrointestinal image to be processed can be reconstructed based on the deep learning model, so as to obtain a higher quality reconstructed image, so that the doctor or recognition model can identify it based on the reconstructed image. This solution uses some labeled image data and a large amount of unlabeled image data to train the deep learning model, thereby improving the quality of images reconstructed by the deep learning model, and can more accurately identify gastrointestinal images based on the reconstructed images.

[0072] Based on the above embodiments, the present application also provides an electronic device, comprising: a memory and at least one processor; the memory is used to store computer execution instructions; the at least one processor is used to execute the computer execution instructions stored in the memory, so that the at least one processor executes the method described in the above embodiments.

[0073] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned data processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0074] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0079] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0080] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0082] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A gastrointestinal image reconstruction method based on semi-supervised learning, characterized in that: The method comprises: Acquire gastrointestinal imaging-related image data, divide the image data into labeled first image data and unlabeled second image data, and filter the unlabeled second image data according to a linear filter; Performing supervised training on a deep learning model based on the labeled first image data, wherein the deep learning model is used to reconstruct the labeled first image data to obtain a first reconstructed image, so as to perform supervised training based on a difference between data of the first reconstructed image and the labeled first image data; Performing unsupervised training on a deep learning model based on the filtered unlabeled second image data, wherein the deep learning model is used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and performing unsupervised training based on a difference between data of the second reconstructed image and the unlabeled second image data; Obtain the gastrointestinal image to be processed and input it into the trained deep learning model to determine the reconstructed image. The step of filtering the unlabeled second image data with a linear filter comprises: Obtaining the pixel value and pixel coordinates of each pixel point of the unlabeled second image data; Filtering the pixel value of the pixel point according to the pixel value, the pixel coordinates, and the pixel values ​​of the neighboring pixels of the pixel point to determine the filtered unlabeled second image data, The filtered unlabeled second image data is determined according to the following formula: ， Wherein, L represents the total number of pixels in the filtered unlabeled second image data, The pixel coordinates are The pixel value of the pixel at the position, represents the Laplace function, is the direction parameter of the Laplace function, is with The coordinates of the pixels adjacent to the position are The pixel values ​​of the adjacent pixels, Q represents the linear filter function, represents the linear filter factor parameter, , is the smoothing factor parameter and satisfies , represents the unlabeled second image data after filtering, The deep learning model includes an encoder and a decoder, the encoder includes multiple encoding layers, the encoding layers include convolutional layers and pooling layers, and the decoder includes multiple decoding layers. The step of performing unsupervised training on the deep learning model based on the filtered unlabeled second image data includes: Extracting image features of the filtered unlabeled second image data through an encoding layer of the encoder to obtain output features of the encoder; The output features of the encoder are processed by the decoding layer of the decoder to obtain the output features of the decoder; Reconstructing the image according to the output features of the decoder to obtain a second reconstructed image; According to the difference between the data of the second reconstructed image and the unlabeled second image data and the difference between the output features of the encoder and the output features of the decoder, determining the adjustment of the parameters of the deep learning model to perform unsupervised training on the deep learning model, The image features of the filtered unlabeled second image data are extracted through the encoding layer of the encoder, and the output features of the encoder are obtained according to the following formula: ， in, represents the unlabeled second image data after filtering, is the output feature of the encoder at layer l, is the encoder convolution kernel of layer l, is the encoder bias term of the lth layer, represents the average pooling operation of the pooling layer, represents the convolution operation of the convolutional layer, Among them, the output features of the encoder are processed by the decoding layer of the decoder, and the output features of the decoder are determined according to the following formula: ， in, is the output feature of the encoder at layer l, is the output feature of the decoder at layer l, represents the upsampling operation, represents the transposed convolution operation, represents the decoder convolution kernel of layer l, is the decoder bias term of layer l, Wherein, an optimizer is also included between the encoder and the decoder, and the optimizer is used to adjust the encoder parameters of the next layer according to the output characteristics of the encoder and the output characteristics of the decoder to determine the output characteristics of the adjusted encoder; The optimizer determines the output characteristics of the adjusted encoder according to the following formula: ， in, represents the optimizer feedback factor of the lth layer, represents the gradient operation, Represents the characteristic residual function, which is used to calculate and The characteristic differences between represents the output features of the encoder of the l+1th layer after passing through the optimizer, represents the learning rate, The difference between the data of the second reconstructed image and the unlabeled second image data is determined according to the following formula: ， Wherein, M represents the difference between the data of the second reconstructed image and the unlabeled second image data, that is, the loss function, is the filtered unlabeled second image data, is the data of the second reconstructed image, N represents the total number of features of the encoder and decoder, represents the convolution operation of the convolutional layer, is the dynamic adjustment factor of the loss function, is the output feature of the decoder at layer l, is the output feature of the encoder at layer l, express norm, in, , represents the tanh function, Represents a transposed convolution operation.

2. A gastrointestinal image reconstruction device based on semi-supervised learning, using the gastrointestinal image reconstruction method based on semi-supervised learning according to claim 1, characterized in that: The device comprises: An image data acquisition module, used for acquiring image data related to gastrointestinal imaging, dividing the image data into first image data with annotations and second image data without annotations, and filtering the second image data without annotations according to a linear filter; A first model training module, used for performing supervised training on a deep learning model based on the labeled first image data, wherein the deep learning model is used for reconstructing the labeled first image data to obtain a first reconstructed image, so as to perform supervised training based on a difference between data of the first reconstructed image and the labeled first image data; A second model training module is used to perform unsupervised training on a deep learning model based on the filtered unlabeled second image data, wherein the deep learning model is used to reconstruct the unlabeled second image data to obtain a second reconstructed image, and perform unsupervised training based on the difference between the data of the second reconstructed image and the unlabeled second image data; The reconstructed image generation module is used to obtain the gastrointestinal image to be processed and input it into the trained deep learning model to determine the reconstructed image.

3. An electronic device, characterized in that include: memory and at least one processor; The memory is used to store computer-executable instructions; The at least one processor is configured to execute computer-executable instructions stored in the memory, so that the at least one processor performs the method of claim 1 .

Images