Endoscopic Image Processing System and Method Based on Model Analysis

Abstract

The present invention discloses an endoscopic image processing system and method based on model analysis. By acquiring endoscopic video stream data, key frames are extracted to form an image set, and then shape, contour, and color feature extraction are performed on each image. Similarity analysis is carried out using preset lesion image features, and the image set is segmented and clustered to obtain multiple clustering groups. The target clustering group is screened through DCA feature fusion and similarity threshold, and histogram equalization enhancement is performed on it. Finally, the target and non-target clustering group images are integrated to generate a recommended image set, and multiple recommended sets are generated based on various preset lesion features to assist in diagnostic analysis. Through the present invention, it is possible to assist medical staff in quickly diagnosing and analyzing endoscopic images of different diseases, effectively improving the efficiency of endoscopic analysis.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and more specifically, to an endoscopic image processing system and method based on model analysis. Background Art

[0002] Endoscopic examination is an indispensable part of modern medical diagnosis, but the complexity and diversity of endoscopic images pose extremely high requirements for doctors. Traditional image processing methods often have poor effects when facing complex lesions, especially in image segmentation, feature extraction, and accurate recognition. In traditional technologies, pure manual analysis and lesion annotation are often relied on, and it is necessary to manually view the lesion sites of images, which greatly depends on manual experience and does not fully utilize information technology means to assist in the analysis of endoscopic images, resulting in low efficiency of lesion analysis based on endoscopic images. Therefore, it has become an urgent need to develop a method that can effectively process endoscopic images and assist doctors in making quick and accurate diagnoses. Summary of the Invention

[0003] The present invention overcomes the defects of the prior art and provides an endoscopic image processing system and method based on model analysis.

[0004] The first aspect of the present invention provides an endoscopic image processing method based on model analysis, including:

[0005] Obtaining endoscopic video stream data of a target user, extracting key frames from the video stream data to obtain an endoscopic image set;

[0006] Extracting features of each image in the endoscopic image set based on shape, contour, and color to obtain image feature data;

[0007] Obtaining various preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, and based on the similarity, performing image segmentation processing on each image in the endoscopic image set to obtain a second image set;

[0008] Taking the image feature data in the second image set as clustering samples, clustering each image in the second image set based on the DBSCAN clustering model to obtain a clustering result;

[0009] Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering groups to form fusion feature data, performing feature similarity analysis on the fusion feature data and a preset lesion image feature, and screening out target clustering groups based on a preset similarity threshold;

[0010] Enhance the features of the images in the target clustering group based on histogram equalization, and integrate the images in the target clustering group and the non-target clustering group to form a recommended image set;

[0011] Generate multiple recommended image sets based on multiple preset lesion image features.

[0012] In this solution, the endoscopic video stream data of the target user is obtained, and key frames are extracted from the video stream data to obtain an endoscopic image set. Specifically:

[0013] Obtain the endoscopic video stream data of the target user;

[0014] Extract key frames from the video stream data and record the corresponding time information to form an endoscopic image set.

[0015] In this solution, for each image in the endoscopic image set, feature extraction based on shape, contour, and color is performed to obtain image feature data. Specifically:

[0016] Perform image denoising and standardization preprocessing on the endoscopic image set;

[0017] Based on the three dimensions of shape, contour, and color, perform feature extraction on each image in the endoscopic image set. For the shape and contour dimensions, feature extraction is performed based on edge detection operators, and for the color dimension, feature extraction is performed based on color moment calculation, and feature data for the three dimensions are obtained;

[0018] Vectorize the feature data for the three dimensions to form the image feature data for each image.

[0019] In this solution, multiple preset lesion image features are obtained, feature similarity analysis is performed on the image feature data and a preset lesion image feature, and based on the similarity, image segmentation processing is performed on each image in the endoscopic image set, and a second image set is obtained. Specifically:

[0020] Obtain multiple preset lesion image features from the system database;

[0021] Mark a preset lesion image feature and vectorize the preset lesion image feature to form a comparison feature;

[0022] In the endoscopic image set, divide each image into multiple image regions, obtain the feature vectors corresponding to the multiple image regions through the image feature data of each image, perform similarity analysis on the feature vectors and the comparison feature based on the standard Euclidean distance, and filter out the highly similar image regions based on the first preset similarity;

[0023] Segment and extract the highly similar image regions of each image, and integrate the extracted images to form a second image set.

[0024] In this solution, the image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result. Specifically:

[0025] Use the image feature data of each image in the second image set as clustering sample data;

[0026] Based on the DBSCAN clustering model, cluster the clustering sample data. Before clustering, set the radiation radius of sample points. During the clustering process, calculate the similarity between sample data through the standard Euclidean distance and obtain the clustering result;

[0027] In the clustering result, it includes multiple clustering groups divided from the second image set.

[0028] In this solution, reorder the second image set according to the clustering result. Based on the clustering result, obtain multiple clustering groups, perform feature fusion based on DCA on the image features in the clustering groups to form fused feature data, perform feature similarity analysis on the fused feature data and a preset lesion image feature, and based on the preset similarity threshold, screen out the target clustering groups. Specifically:

[0029] Obtain multiple clustering groups through the clustering result;

[0030] Set the clustering group order. Based on the clustering group order, reorder the second image set. After reordering, the second image set includes multiple segments of image sets, and each segment of image set corresponds to all the images in a clustering group;

[0031] Take a clustering group as an analysis unit, perform feature fusion based on DCA on the image feature data of each image in the clustering group. During the fusion process, first generate corresponding feature maps based on the image feature data of each image, input the feature maps into a preset neural network for weighted fusion, and generate fused feature data;

[0032] Perform feature vectorization and feature similarity calculation on the fused feature data and the preset lesion image feature. The similarity calculation is based on the standard Euclidean distance method. Compare the similarity calculation result with the preset similarity threshold. If it is higher than the preset similarity threshold, mark it as the target clustering group;

[0033] Judge multiple clustering groups and screen out all target clustering groups.

[0034] In this solution, the images in the target clustering group are feature-enhanced based on histogram equalization, and the images in the target clustering group and the non-target clustering group are integrated to form a recommended image set. Specifically:

[0035] Calculate the image histogram and cumulative distribution function for each image in the target clustering group, and perform image enhancement on each image through histogram equalization to obtain the enhanced image;

[0036] Based on the position of each image in the target clustering group in the second image set, replace the images with the enhanced images to form the enhanced second image set;

[0037] In the enhanced second image set, label the images in the target clustering group and the non-target clustering group to form a recommended image set;

[0038] Send the recommended image set to a preset display terminal.

[0039] The second aspect of the present invention also provides an endoscope image processing system based on model analysis. The system includes: a memory and a processor. The memory includes an endoscope image processing program based on model analysis. When the endoscope image processing program based on model analysis is executed by the processor, the following steps are implemented:

[0040] Obtain the endoscope video stream data of the target user, and extract key frames from the video stream data to obtain an endoscope image set;

[0041] Extract the features of shape, contour, and color for each image in the endoscope image set to obtain image feature data;

[0042] Obtain multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, and based on the similarity, perform image segmentation processing on each image in the endoscope image set to obtain a second image set;

[0043] Use the image feature data in the second image set as clustering samples, and perform clustering on each image in the second image set based on the DBSCAN clustering model to obtain a clustering result;

[0044] Reorder the second image set according to the clustering result, obtain multiple clustering groups based on the clustering result, perform feature fusion of the image features in the clustering groups based on DCA to form fusion feature data, perform feature similarity analysis on the fusion feature data and a preset lesion image feature, and filter out the target clustering group based on a preset similarity threshold;

[0045] Enhance the features of the images in the target clustering group based on histogram equalization, and integrate the images in the target clustering group and the non-target clustering group to form a recommended image set;

[0046] Generate multiple recommended image sets based on multiple preset lesion image features.

[0047] The third aspect of the present invention also provides a computer-readable storage medium, which includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by a processor, the steps of the endoscopic image processing method based on model analysis as described in any one of the above are implemented.

[0048] The present invention discloses an endoscopic image processing system and method based on model analysis. By acquiring endoscopic video stream data, key frames are extracted to form an image set, and then the shape, contour, and color features of each image are extracted. Similarity analysis is performed using preset lesion image features, and the image set is segmented and clustered to obtain multiple clustering groups. The target clustering group is screened through DCA feature fusion and similarity threshold, and histogram equalization enhancement is performed on it. Finally, the images of the target and non-target clustering groups are integrated to generate a recommended image set, and multiple recommended sets are generated based on multiple preset lesion features to assist in diagnostic analysis. Through the present invention, it is possible to assist medical staff in quickly diagnosing and analyzing endoscopic images of different diseases, effectively improving the efficiency of endoscopic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Shows a flowchart of an endoscopic image processing method based on model analysis of the present invention;

[0050] Figure 2 Shows a block diagram of an endoscopic image processing system based on model analysis of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] In order to more clearly understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0052] Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention may be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0053] Figure 1 Shows a flowchart of an endoscopic image processing method based on model analysis of the present invention.

[0054] As Figure 1 shown, the first aspect of the present invention provides an endoscopic image processing method based on model analysis, including:

[0055] S102, obtaining the endoscopic video stream data of the target user, extracting key frames from the video stream data to obtain an endoscopic image set;

[0056] S104, extracting features of each image in the endoscopic image set based on shape, contour, and color to obtain image feature data;

[0057] S106, obtaining multiple preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, and based on the similarity, performing image segmentation processing on each image in the endoscopic image set to obtain a second image set;

[0058] S108, using the image feature data in the second image set as clustering samples, clustering each image in the second image set based on the DBSCAN clustering model to obtain a clustering result;

[0059] S110, reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering groups to form fusion feature data, performing feature similarity analysis on the fusion feature data and a preset lesion image feature, and screening out the target clustering group based on a preset similarity threshold;

[0060] S112, enhancing the features of the images in the target clustering group based on histogram equalization, and integrating the images in the target clustering group and non-target clustering groups to form a recommended image set;

[0061] S114, generating multiple recommended image sets based on multiple preset lesion image features.

[0062] It should be noted that the target user is generally a patient.

[0063] According to an embodiment of the present invention, the obtaining of the endoscopic video stream data of the target user, extracting key frames from the video stream data to obtain an endoscopic image set is specifically:

[0064] Obtaining the endoscopic video stream data of the target user;

[0065] Extracting key frames from the video stream data and recording the corresponding time information to form an endoscopic image set.

[0066] It should be noted that the endoscopic image set includes multiple images, and these images are generally stored in chronological order or disorderly.

[0067] According to an embodiment of the present invention, the feature extraction based on shape, contour, and color is performed on each image in the endoscopic image set to obtain image feature data, specifically as follows:

[0068] Perform image denoising and normalization preprocessing on the endoscopic image set;

[0069] Based on three dimensions of shape, contour, and color, perform feature extraction on each image in the endoscopic image set. For the shape and contour dimensions, feature extraction is performed based on edge detection operators, and for the color dimension, feature extraction is performed based on color moment calculation, and feature data in the three dimensions are obtained;

[0070] Perform feature vectorization on the feature data in the three dimensions to form the image feature data of each image.

[0071] It should be noted that the features after vectorization of the image feature data are used for subsequent similarity analysis and clustering.

[0072] According to an embodiment of the present invention, the acquisition of multiple preset lesion image features, the feature similarity analysis of the image feature data and a preset lesion image feature, and based on the similarity, perform image segmentation processing on each image in the endoscopic image set to obtain a second image set, specifically as follows:

[0073] Obtain multiple preset lesion image features from the system database;

[0074] Mark a preset lesion image feature, and perform feature vectorization on the preset lesion image feature to form a comparison feature;

[0075] In the endoscopic image set, divide each image into multiple image regions, obtain the feature vectors corresponding to the multiple image regions through the image feature data of each image, perform similarity analysis of the feature vectors and the comparison feature based on the standard Euclidean distance, and screen out the highly similar image regions based on the first preset similarity;

[0076] Perform segmentation extraction on the highly similar image regions of each image, and integrate the extracted images to form a second image set.

[0077] It should be noted that among the multiple preset lesion image features, each preset lesion image feature corresponds to the endoscopic image feature of a disease, and this feature is used for comparative analysis.

[0078] In the division of multiple image regions for each image, specifically, perform grid division, and the number of divisions is set by the user or the system. The preset similarity threshold and the first preset similarity are both set by the user.

[0079] According to an embodiment of the present invention, taking the image feature data in the second image set as clustering samples, and clustering each image in the second image set based on the DBSCAN clustering model to obtain a clustering result, specifically:

[0080] Taking the image feature data of each image in the second image set as clustering sample data;

[0081] Based on the DBSCAN clustering model, clustering the clustering sample data. Before clustering, set the radiation radius of sample points. During the clustering process, calculate the similarity between sample data through the standard Euclidean distance, and obtain the clustering result;

[0082] In the clustering result, it includes multiple clustering groups divided from the second image set.

[0083] It should be noted that the image feature data of each image in the second image set is specifically obtained by acquiring the image features of the corresponding region based on the segmentation result in the image feature data of the endoscopic image set. The image feature data of the endoscopic image set includes the feature data of the entire image. The clustering result is used to store the division information, and corresponding multiple clustering groups can be obtained through the clustering result. In the multiple clustering groups, each clustering group includes at least one image.

[0084] According to an embodiment of the present invention, reordering the second image set based on the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fusion feature data, and performing feature similarity analysis on the fusion feature data and a preset lesion image feature, and screening out the target clustering group based on a preset similarity threshold, specifically:

[0085] Obtaining multiple clustering groups through the clustering result;

[0086] Setting the clustering group order, and reordering the second image set based on the clustering group order. After reordering, the second image set includes multiple segments of image sets, and each segment of image set corresponds to all the images in one clustering group;

[0087] Taking one clustering group as an analysis unit, performing feature fusion based on DCA on the image feature data of each image in the one clustering group. During the fusion process, first generate a corresponding feature map based on the image feature data of each image, input the feature map into a preset neural network for weighted fusion, and generate fusion feature data;

[0088] Performing feature vectorization and feature similarity calculation on the fusion feature data and the one preset lesion image feature. The similarity calculation is based on the standard Euclidean distance method. Comparing the similarity calculation result with the preset similarity threshold. If it is higher than the preset similarity threshold, it is marked as the target clustering group;

[0089] Judge multiple clustering groups and screen out all target clustering groups.

[0090] It should be noted that after reordering, images with similar lesion characteristics can be classified and sorted, improving the auxiliary decision-making effect based on subsequent lesion analysis. The order of the set clustering groups can be based on a random order. The preset neural network is a convolutional neural network. DCA is the Dynamic Class Activation feature fusion algorithm.

[0091] According to an embodiment of the present invention, enhancing the features of the images in the target clustering group based on histogram equalization, and integrating the images in the target clustering group and the non-target clustering group to form a recommended image set, specifically:

[0092] Calculate the image histogram and cumulative distribution function for each image in the target clustering group, and perform image enhancement on each image through histogram equalization to obtain the enhanced image;

[0093] Based on the position of each image in the target clustering group in the second image set, replace the images with the enhanced images to form an enhanced second image set;

[0094] In the enhanced second image set, label the images in the target clustering group and the non-target clustering group to form a recommended image set;

[0095] Send the recommended image set to a preset display terminal.

[0096] It should be noted that the second image set includes the target clustering group and the non-target clustering group. When labeling the images in the target clustering group and the non-target clustering group, the images in the target clustering group are images with certain image enhancement processing, which is convenient for users to perform quick analysis and classification, and the image set of the target clustering group is the atlas closest to the preset disease characteristics.

[0097] In generating multiple recommended image sets based on multiple preset lesion image features, each preset lesion image feature corresponds to generating a recommended image set, and the analysis process is the same, all of which are analyzed through the same endoscopic image set. Among the multiple preset lesion image features, they are obtained by feature extraction from historical images or manually marked lesion images and used as real-time analysis and comparison features.

[0098] In traditional technologies, it is generally difficult to distinguish the features of endoscopic images, and manual experience is required for analysis and lesion annotation. The process relies heavily on manual experience to a large extent, and the existing technologies lack an effective sorting and classification process for endoscopic images, with insufficient information integration capabilities. To solve the above problems, the present invention performs disease similarity analysis and effective segmentation on endoscopic images to form a set of segmented images. This process can effectively extract useful features for manual screening and analysis. Then, the image set is clustered and grouped, and the image set is reordered based on the grouping results to form a recommended image set. Moreover, the present invention generates corresponding recommended image sets for each disease for medical professionals to conduct reasonable analysis. In the recommended image set, the high-feature and low-feature images are effectively reordered, and the images with similar features are arranged in an orderly manner, effectively assisting doctors in making quick and accurate diagnoses. Moreover, based on user needs, different recommended image sets can be generated for different diseases to assist doctors in quickly analyzing the disease images, greatly improving the efficiency of endoscopic analysis.

[0099] According to an embodiment of the present invention, it further includes:

[0100] Obtain a preset lesion image feature;

[0101] Obtain the sample image to which the preset lesion image feature belongs from the system database;

[0102] Standardize the sample image and perform feature extraction based on the features of shape, contour, and color to obtain sample feature data;

[0103] Construct a GAN-based generation network, where the generation network includes a generator, a discriminator, and a preset loss function;

[0104] Import the sample feature data as real data into the generation network, and the generator performs feature learning on the real data and generates simulated feature data;

[0105] Based on the discriminator, perform data judgment on the simulated feature data, and optimize the generator and the discriminator through the judgment result and the preset loss function;

[0106] Based on the generator and the discriminator, perform adversarial training in a loop until the generator and the discriminator reach Nash equilibrium;

[0107] Based on the trained generator, generate a preset amount of simulated images;

[0108] Calculate the image histogram and cumulative distribution function of the simulated images, and perform image enhancement on the simulated images through histogram equalization to obtain enhanced simulated images;

[0109] Perform feature extraction on the enhanced simulated images and form a second preset lesion image feature.

[0110] It should be noted that the second preset lesion image feature is supplementary data of a preset lesion image feature and can be directly used as comparison data. In the image recognition task of endoscopic images, due to the limited amount of feature data in the database, there is often a situation where the amount of comparison feature data for a certain disease is small. When performing image analysis and recognition, the recognition rate of this disease is low and the effect of feature extraction and analysis is poor. Therefore, in the present invention, by obtaining the original feature image, generating simulated features based on the GAN model, finally enhancing the image features of the simulated data through image enhancement, and importing the simulated data as supplementary data into the corresponding database, efficient endoscopic image feature recognition and analysis can be achieved when the amount of comparison sample data is small, the diagnostic decision-making efficiency can be improved, and the recognition and analysis efficiency of multiple diseases can be effectively improved.

[0111] Figure 2 Fig. shows a block diagram of an endoscopic image processing system based on model analysis according to the present invention.

[0112] In a second aspect of the present invention, there is also provided an endoscopic image processing system 2 based on model analysis. The system includes: a memory 21 and a processor 22. The memory includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by the processor, the following steps are implemented:

[0113] Obtain the endoscopic video stream data of the target user, extract key frames from the video stream data to obtain an endoscopic image set;

[0114] Extract features based on shape, contour, and color for each image in the endoscopic image set to obtain image feature data;

[0115] Obtain multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, and based on the similarity, perform image segmentation processing on each image in the endoscopic image set to obtain a second image set;

[0116] Use the image feature data in the second image set as clustering samples, and perform clustering on each image in the second image set based on the DBSCAN clustering model to obtain a clustering result;

[0117] Reorder the second image set according to the clustering result, obtain multiple clustering groups based on the clustering result, perform feature fusion based on DCA on the image features in the clustering groups to form fusion feature data, perform feature similarity analysis on the fusion feature data and a preset lesion image feature, and based on a preset similarity threshold, screen out the target clustering group;

[0118] Enhance the features of the images in the target clustering group based on histogram equalization, and form a recommended image set by integrating the images in the target clustering group and the non-target clustering group;

[0119] Generate multiple recommended image sets based on multiple preset lesion image features.

[0120] It should be noted that the target user is generally a patient.

[0121] According to an embodiment of the present invention, the obtaining of the endoscopic video stream data of the target user and the extraction of key frames from the video stream data to obtain an endoscopic image set are specifically as follows:

[0122] Obtain the endoscopic video stream data of the target user;

[0123] Extract key frames from the video stream data and record the corresponding time information to form an endoscopic image set.

[0124] It should be noted that the endoscopic image set includes multiple images, and these images are generally stored based on time sequence or stored disorderly.

[0125] According to an embodiment of the present invention, the extraction of feature data based on shape, contour, and color for each image in the endoscopic image set is specifically as follows:

[0126] Perform image denoising and normalization preprocessing on the endoscopic image set;

[0127] Based on three dimensions of shape, contour, and color, extract features for each image in the endoscopic image set. For the shape and contour dimensions, feature extraction is performed based on edge detection operators, and for the color dimension, feature extraction is performed based on color moment calculation, and feature data for the three dimensions are obtained;

[0128] Vectorize the feature data for the three dimensions to form the image feature data for each image.

[0129] It should be noted that the features after vectorizing the image feature data are used for subsequent similarity analysis and clustering.

[0130] According to an embodiment of the present invention, the obtaining of multiple preset lesion image features, the feature similarity analysis of the image feature data with a preset lesion image feature, and the image segmentation processing of each image in the endoscopic image set based on the similarity to obtain a second image set are specifically as follows:

[0131] Obtain multiple preset lesion image features from the system database;

[0132] Mark a preset lesion image feature, and vectorize the preset lesion image feature to form a comparison feature;

[0133] In the endoscopic image set, multiple image regions are divided for each image. Through the image feature data of each image, feature vectors corresponding to the multiple image regions are obtained. The similarity analysis of the feature vectors and the comparison features is performed based on the standard Euclidean distance. Based on the first preset similarity, highly similar image regions are screened out;

[0134] The highly similar image regions of each image are segmented and extracted, and the extracted images are integrated to form a second image set.

[0135] It should be noted that among the multiple preset lesion image features, each preset lesion image feature corresponds to the endoscopic image feature of a disease, and this feature is used for comparative analysis.

[0136] Among the multiple image regions divided for each image, specifically, grid division is performed, and the number of divisions is set by the user or the system. The preset similarity threshold and the first preset similarity are both set by the user.

[0137] According to an embodiment of the present invention, taking the image feature data in the second image set as clustering samples, clustering is performed on each image in the second image set based on the DBSCAN clustering model to obtain a clustering result, specifically:

[0138] Taking the image feature data of each image in the second image set as clustering sample data;

[0139] Based on the DBSCAN clustering model, clustering is performed on the clustering sample data. Before clustering, the radiation radius of the sample points is set. During the clustering process, the similarity between the sample data is calculated through the standard Euclidean distance, and a clustering result is obtained;

[0140] In the clustering result, it includes multiple clustering groups divided for the second image set.

[0141] It should be noted that the image feature data of each image in the second image set is specifically obtained by obtaining the image features of the corresponding regions based on the segmentation result in the image feature data of the endoscopic image set. The image feature data of the endoscopic image set includes the feature data of the entire image. The clustering result is used to store the division information, and the corresponding multiple clustering groups can be obtained through the clustering result. Among the multiple clustering groups, each clustering group includes at least one image.

[0142] According to an embodiment of the present invention, reordering the second image set through the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering groups to form fusion feature data, and performing feature similarity analysis on the fusion feature data and a preset lesion image feature, and screening out the target clustering group based on the preset similarity threshold, specifically:

[0143] Obtain multiple clustering groups from the clustering results;

[0144] Set the order of the clustering groups. Based on the order of the clustering groups, reorder the second image set. After reordering, the second image set includes multiple segments of image sets, and each segment of image set corresponds to all the images in one clustering group;

[0145] Taking one clustering group as the analysis unit, perform feature fusion based on DCA on the image feature data of each image in the one clustering group. During the fusion process, first generate corresponding feature maps based on the image feature data of each image, input the feature maps into a preset neural network for weighted fusion, and generate fused feature data;

[0146] Perform feature vectorization and feature similarity calculation on the fused feature data and the preset lesion image feature. The similarity calculation is based on the standard Euclidean distance method. Compare the similarity calculation result with a preset similarity threshold. If it is higher than the preset similarity threshold, mark it as the target clustering group;

[0147] Judge multiple clustering groups and screen out all target clustering groups.

[0148] It should be noted that after reordering, images with similar lesion characteristics can be classified and sorted, improving the auxiliary decision-making effect in subsequent lesion-based analysis. The setting of the order of the clustering groups can be based on a random order. The preset neural network is a convolutional neural network. DCA is the Dynamic Class Activation feature fusion algorithm.

[0149] According to an embodiment of the present invention, perform feature enhancement on the images in the target clustering group based on histogram equalization, and perform image integration on the images in the target clustering group and the non-target clustering group to form a recommended image set, specifically:

[0150] Calculate the image histogram and cumulative distribution function for each image in the target clustering group, and perform image enhancement on each image through histogram equalization to obtain the enhanced image;

[0151] Based on the position of each image in the target clustering group in the second image set, perform image replacement on the enhanced images to form the enhanced second image set;

[0152] In the enhanced second image set, label the images in the target clustering group and the non-target clustering group to form a recommended image set;

[0153] Send the recommended image set to a preset display terminal.

[0154] It should be noted that the second image set includes a target clustering group and a non-target clustering group. When annotating the images in the target clustering group and the non-target clustering group, the images in the target clustering group are images with a certain degree of image enhancement processing, which is convenient for users to perform quick analysis and classification. Moreover, the image set of the target clustering group is the image set closest to the preset disease characteristics.

[0155] In the generation of multiple recommended image sets based on multiple preset lesion image characteristics, each preset lesion image characteristic corresponds to a generated recommended image set, and the analysis process is the same. All are analyzed through the same endoscopic image set. Among the multiple preset lesion image characteristics, they are obtained by extracting features from historical images or manually marked lesion images and used as real-time analysis and comparison features.

[0156] In traditional technologies, endoscopic image characteristics are generally difficult to distinguish, and manual experience is required for analysis and lesion annotation. The process depends to a large extent on manual experience, and the existing technologies lack an effective sorting and classification process for endoscopic images, with insufficient information integration ability. To solve the above problems, the present invention performs disease similarity analysis and effective segmentation on endoscopic images to form a segmented image set. This process can effectively extract useful features for manual screening and analysis. Then, the image set is clustered and grouped, and the image set is reordered through the grouping results to form a recommended image set. Moreover, the present invention generates corresponding recommended image sets based on each disease for medical professionals to perform reasonable analysis. In the recommended image set, the high-feature and low-feature images are effectively reordered, and the images with similar features are arranged in an orderly combination, effectively assisting doctors in making quick and accurate diagnoses. Moreover, based on user needs, different recommended image sets can be generated for different diseases to assist doctors in quickly analyzing disease images, greatly improving the efficiency of endoscopic analysis.

[0157] The third aspect of the present invention also provides a computer-readable storage medium, which includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by a processor, the steps of the endoscopic image processing method based on model analysis as described in any one of the above are implemented.

[0158] The present invention discloses an endoscope image processing system and method based on model analysis. By acquiring endoscope video stream data, key frames are extracted to form an image set, and then shape, contour, and color features are extracted from each image. Similarity analysis is performed using preset lesion image features, and the image set is segmented and clustered to obtain multiple clustering groups. The target clustering group is screened through DCA feature fusion and similarity threshold, and histogram equalization enhancement is performed on it. Finally, the target and non-target clustering group images are integrated to generate a recommended image set, and multiple recommended sets are generated based on various preset lesion features to assist in diagnostic analysis. Through the present invention, it is possible to assist medical staff in quickly diagnosing and analyzing endoscope images for different diseases, effectively improving the efficiency of endoscope analysis. In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the couplings, direct couplings, or communication connections between the various components shown or discussed may be through some interfaces, and the indirect couplings or communication connections of devices or units may be electrical, mechanical, or other forms.

[0159] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they may be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, in each embodiment of the present invention, the various functional units can all be integrated in one processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit; the above-mentioned integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0161] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as removable storage devices, read-only memory (ROM, Read-On l y Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical disks that can store program codes.

[0162] Alternatively, if the above-integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

[0163] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention.

Claims

1. An endoscopic image processing method based on model analysis, characterized in that: include: Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set; Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data; Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set; The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result; Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold; Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group; Based on multiple preset lesion image features, multiple recommended image sets are generated.

2. The endoscopic image processing method based on model analysis according to claim 1, characterized in that: The step of obtaining the endoscope video stream data of the target user and extracting key frames from the video stream data to obtain an endoscope image set is as follows: Obtain the endoscope video stream data of the target user; The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

3. The endoscopic image processing method based on model analysis according to claim 2, characterized in that: The feature extraction based on shape, contour and color is performed on each image in the endoscope image set to obtain image feature data, specifically: The endoscopic image set was preprocessed for image noise reduction and standardization; Based on the three dimensions of shape, contour and color, feature extraction is performed on each image in the endoscope image set. The shape and contour dimensions are based on edge detection operators, and the color dimension is based on color moment calculation to obtain feature data of the three dimensions. The feature data of the three dimensions are vectorized to form image feature data of each image.

4. The endoscopic image processing method based on model analysis according to claim 3 is characterized in that: The method of obtaining a plurality of preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, performing image segmentation processing on each image in the endoscopic image set based on the similarity, and obtaining a second image set is specifically as follows: Acquire multiple preset lesion image features from the system database; Marking a preset lesion image feature, and performing feature vectorization on the preset lesion image feature to form a contrast feature; In the endoscope image set, each image is divided into multiple image regions, feature vectors corresponding to the multiple image regions are obtained through image feature data of each image, similarity analysis is performed on the feature vectors and the contrast features based on the standard Euclidean distance, and image regions with high similarity are screened out based on a first preset similarity; The image regions with high similarity of each image are segmented and extracted, and the extracted images are integrated to form a second image set.

5. The method for endoscopic image processing based on model analysis according to claim 4, characterized in that: The image feature data in the second image set is used as a clustering sample, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result, which is specifically: Using image feature data of each image in the second image set as clustering sample data; Based on the DBSCAN clustering model, cluster sample data are clustered. The radiation radius of sample points is set before clustering. During the clustering process, the similarity between sample data is calculated through the standard Euclidean distance and the clustering results are obtained. The clustering result includes a plurality of clustering groups divided from the second image set.

6. The method for endoscopic image processing based on model analysis according to claim 5, characterized in that: The second image set is reordered according to the clustering result, a plurality of cluster groups are obtained based on the clustering result, image features in the cluster groups are subjected to feature fusion based on DCA to form fused feature data, the fused feature data is subjected to feature similarity analysis with a preset lesion image feature, and a target cluster group is screened out based on a preset similarity threshold, specifically: Obtain multiple clustering groups through clustering results; The clustering group sequence is set, and the second image set is reordered based on the clustering group sequence. After the reordering, the second image set includes a plurality of image sets, and each image set corresponds to all images in a clustering group. Taking a cluster group as an analysis unit, the image feature data of each image in the cluster group is subjected to feature fusion based on DCA. During the fusion process, a corresponding feature map is first generated based on the image feature data of each image, and the feature map is input into a preset neural network for weighted fusion, and fused feature data is generated; Performing feature vectorization and feature similarity calculation on the fused feature data and the one of the preset lesion image features, the similarity calculation is based on the standard Euclidean distance method, and the similarity calculation result is compared with the preset similarity threshold. If it is higher than the preset similarity threshold, it is marked as a target cluster group; Multiple cluster groups are judged and all target cluster groups are screened out.

7. The method for endoscopic image processing based on model analysis according to claim 6, characterized in that: The feature enhancement of the images in the target cluster group based on histogram equalization and the image integration of the images in the target cluster group and the non-target cluster group to form a recommended image set are specifically as follows: The image histogram and cumulative distribution function are calculated for each image of the target cluster group, and each image is enhanced by histogram equalization to obtain an enhanced image; Based on the position of each image of the target cluster group in the second image set, performing image replacement on the enhanced image to form an enhanced second image set; In the enhanced second image set, images in the target cluster group and the non-target cluster group are annotated to form a recommended image set; The recommended image set is sent to a preset display terminal.

8. An endoscopic image processing system based on model analysis, characterized in that: The system includes: a memory and a processor, wherein the memory includes an endoscopic image processing program based on model analysis, and when the endoscopic image processing program based on model analysis is executed by the processor, the following steps are implemented: Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set; Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data; Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set; The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result; Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold; Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group; Based on multiple preset lesion image features, multiple recommended image sets are generated.

9. The endoscopic image processing system based on model analysis according to claim 8, characterized in that: The step of obtaining the endoscope video stream data of the target user and extracting key frames from the video stream data to obtain an endoscope image set is as follows: Obtain the endoscope video stream data of the target user; The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by a processor, the steps of the endoscopic image processing method based on model analysis as described in any one of claims 1 to 7 are implemented.

Images