Endoscope image processing system and method based on model analysis

By extracting features and clustering data from endoscopic video streams, a recommended image set is generated, overcoming the limitations of traditional endoscopic image processing methods and achieving efficient lesion analysis and rapid diagnosis.

CN118918349AActive Publication Date: 2024-11-08JINAN GUANZE MEDICAL EQUIP CO LTD

Patent Information

Application Number
CN202411191216.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-08
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

Traditional endoscopic image processing methods are ineffective when dealing with complex lesions, especially in terms of image segmentation, feature extraction, and accurate identification. They also rely on manual analysis, resulting in low efficiency in lesion analysis.

Method used

By acquiring endoscopic video stream data, keyframes are extracted and shape, contour, and color features are extracted. Similarity analysis and segmentation are performed using preset lesion image features. The DBSCAN clustering model is applied for clustering. Combined with DCA feature fusion and histogram equalization, a recommended image set is generated.

Benefits of technology

It improves the efficiency of endoscopic image analysis, assists doctors in making rapid and accurate lesion diagnoses, reduces reliance on human experience, and enables effective image sorting and classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918349A_ABST
    Figure CN118918349A_ABST
Patent Text Reader

Abstract

The invention discloses an endoscope image processing system and method based on model analysis, and the method comprises the steps: obtaining endoscope video stream data, extracting key frames to form an image set, and then carrying out the shape, contour and color feature extraction of each image. Performing similarity analysis by using preset lesion image features, and performing segmentation and clustering processing on the image set to obtain a plurality of clustering groups; a target cluster group is screened through DCA feature fusion and a similarity threshold value, and histogram equalization enhancement is carried out on the target cluster group; and finally, integrating the target and non-target cluster group images to generate a recommendation image set, and generating a plurality of recommendation sets according to a plurality of preset lesion features to assist in diagnostic analysis. According to the invention, the medical staff can be assisted to carry out rapid endoscope image diagnosis analysis on different diseases, and the endoscope analysis efficiency is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and more specifically, to an endoscopic image processing system and method based on model analysis. Background Art

[0002] Endoscopic examination is an indispensable part of modern medical diagnosis, but the complexity and diversity of endoscopic images place extremely high demands on doctors. Traditional image processing methods often have poor results when faced with complex lesions, especially in terms of image segmentation, feature extraction, and accurate recognition. In traditional technologies, pure manual analysis and lesion annotation are often relied on, and the lesion site of the image needs to be manually checked. It relies heavily on manual experience and does not make full use of information technology to assist in the analysis of endoscopic images, resulting in low efficiency of lesion analysis based on endoscopic images. Therefore, it is urgent to develop a method that can effectively process endoscopic images and assist doctors in making quick and accurate diagnoses. Summary of the invention

[0003] The present invention overcomes the defects of the prior art and proposes an endoscopic image processing system and method based on model analysis.

[0004] A first aspect of the present invention provides an endoscopic image processing method based on model analysis, comprising:

[0005] Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set;

[0006] Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data;

[0007] Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set;

[0008] The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result;

[0009] Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold;

[0010] Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group;

[0011] Based on multiple preset lesion image features, multiple recommended image sets are generated.

[0012] In this solution, the endoscope video stream data of the target user is obtained, and key frames are extracted from the video stream data to obtain an endoscope image set, specifically:

[0013] Obtain the endoscope video stream data of the target user;

[0014] The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

[0015] In this solution, the feature extraction based on shape, contour and color is performed on each image in the endoscope image set to obtain image feature data, specifically:

[0016] The endoscopic image set was preprocessed for image noise reduction and standardization;

[0017] Based on the three dimensions of shape, contour and color, feature extraction is performed on each image in the endoscope image set. The shape and contour dimensions are based on edge detection operators, and the color dimension is based on color moment calculation to obtain feature data of the three dimensions.

[0018] The feature data of the three dimensions are vectorized to form image feature data of each image.

[0019] In this solution, the method obtains multiple preset lesion image features, performs feature similarity analysis on the image feature data and a preset lesion image feature, and performs image segmentation processing on each image in the endoscopic image set based on the similarity, and obtains the second image set, specifically:

[0020] Acquire multiple preset lesion image features from the system database;

[0021] Marking a preset lesion image feature, and performing feature vectorization on the preset lesion image feature to form a contrast feature;

[0022] In the endoscope image set, each image is divided into multiple image regions, feature vectors corresponding to the multiple image regions are obtained through image feature data of each image, similarity analysis is performed on the feature vectors and the contrast features based on the standard Euclidean distance, and image regions with high similarity are screened out based on a first preset similarity;

[0023] The image regions with high similarity of each image are segmented and extracted, and the extracted images are integrated to form a second image set.

[0024] In this solution, the image feature data in the second image set is used as a clustering sample, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result, which is specifically:

[0025] Using image feature data of each image in the second image set as clustering sample data;

[0026] Based on the DBSCAN clustering model, cluster sample data are clustered. The radiation radius of sample points is set before clustering. During the clustering process, the similarity between sample data is calculated through the standard Euclidean distance and the clustering results are obtained.

[0027] The clustering result includes a plurality of clustering groups divided from the second image set.

[0028] In this solution, the second image set is reordered according to the clustering results, multiple cluster groups are obtained based on the clustering results, the image features in the cluster groups are subjected to feature fusion based on DCA to form fused feature data, the fused feature data is subjected to feature similarity analysis with a preset lesion image feature, and the target cluster group is screened out based on a preset similarity threshold, specifically:

[0029] Obtain multiple clustering groups through clustering results;

[0030] The clustering group sequence is set, and the second image set is reordered based on the clustering group sequence. After the reordering, the second image set includes a plurality of image sets, and each image set corresponds to all images in a clustering group.

[0031] Taking a cluster group as an analysis unit, the image feature data of each image in the cluster group is subjected to feature fusion based on DCA. During the fusion process, a corresponding feature map is first generated based on the image feature data of each image, and the feature map is input into a preset neural network for weighted fusion, and fused feature data is generated;

[0032] Performing feature vectorization and feature similarity calculation on the fused feature data and the one of the preset lesion image features, the similarity calculation is based on the standard Euclidean distance method, and the similarity calculation result is compared with the preset similarity threshold. If it is higher than the preset similarity threshold, it is marked as a target cluster group;

[0033] Multiple cluster groups are judged and all target cluster groups are screened out.

[0034] In this solution, the feature enhancement of the images in the target cluster group is performed based on histogram equalization, and the recommended image set is formed based on the image integration of the images in the target cluster group and the non-target cluster group, specifically:

[0035] The image histogram and cumulative distribution function are calculated for each image of the target cluster group, and each image is enhanced by histogram equalization to obtain an enhanced image;

[0036] Based on the position of each image of the target cluster group in the second image set, performing image replacement on the enhanced image to form an enhanced second image set;

[0037] In the enhanced second image set, images in the target cluster group and the non-target cluster group are annotated to form a recommended image set;

[0038] The recommended image set is sent to a preset display terminal.

[0039] The second aspect of the present invention further provides an endoscopic image processing system based on model analysis, the system comprising: a memory, a processor, the memory comprising an endoscopic image processing program based on model analysis, the endoscopic image processing program based on model analysis when executed by the processor implements the following steps:

[0040] Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set;

[0041] Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data;

[0042] Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set;

[0043] The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result;

[0044] Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold;

[0045] Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group;

[0046] Based on multiple preset lesion image features, multiple recommended image sets are generated.

[0047] The third aspect of the present invention also provides a computer-readable storage medium, which includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by a processor, the steps of the endoscopic image processing method based on model analysis as described in any one of the above items are implemented.

[0048] The present invention discloses an endoscopic image processing system and method based on model analysis. The system obtains endoscopic video stream data, extracts key frames to form an image set, and then extracts shape, contour, and color features from each image. Similarity analysis is performed using preset lesion image features, and the image set is segmented and clustered to obtain multiple cluster groups. The target cluster group is screened by DCA feature fusion and similarity threshold, and histogram equalization is performed on it. Finally, the target and non-target cluster group images are integrated to generate a recommended image set, and multiple recommended sets are generated based on a variety of preset lesion features to assist in diagnosis and analysis. The present invention can assist medical staff in quickly performing endoscopic image diagnosis and analysis of different diseases, effectively improving the efficiency of endoscopic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of an endoscopic image processing method based on model analysis of the present invention is shown;

[0050] Figure 2 A block diagram of an endoscopic image processing system based on model analysis of the present invention is shown. DETAILED DESCRIPTION

[0051] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0053] Figure 1 A flow chart of an endoscopic image processing method based on model analysis of the present invention is shown.

[0054] like Figure 1 As shown, the first aspect of the present invention provides an endoscopic image processing method based on model analysis, comprising:

[0055] S102, obtaining endoscope video stream data of a target user, performing key frame extraction on the video stream data, and obtaining an endoscope image set;

[0056] S104, performing feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data;

[0057] S106, obtaining multiple preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, performing image segmentation processing on each image in the endoscope image set based on the similarity, and obtaining a second image set;

[0058] S108, using the image feature data in the second image set as clustering samples, clustering each image in the second image set based on the DBSCAN clustering model to obtain a clustering result;

[0059] S110, reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering groups to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold;

[0060] S112, performing feature enhancement on the images in the target cluster group based on histogram equalization, and performing image integration on the images in the target cluster group and the non-target cluster group to form a recommended image set;

[0061] S114, generating multiple recommended image sets based on multiple preset lesion image features.

[0062] It should be noted that the target users are generally patients.

[0063] According to an embodiment of the present invention, the endoscopic video stream data of the target user is obtained, and key frames are extracted from the video stream data to obtain an endoscopic image set, specifically:

[0064] Obtain the endoscope video stream data of the target user;

[0065] The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

[0066] It should be noted that the endoscopic image set includes multiple images, which are generally stored in time sequence or in disorder.

[0067] According to an embodiment of the present invention, the feature extraction based on shape, contour and color is performed on each image in the endoscopic image set to obtain image feature data, specifically:

[0068] The endoscopic image set was preprocessed for image noise reduction and standardization;

[0069] Based on the three dimensions of shape, contour and color, feature extraction is performed on each image in the endoscope image set. The shape and contour dimensions are based on edge detection operators, and the color dimension is based on color moment calculation to obtain feature data of the three dimensions.

[0070] The feature data of the three dimensions are vectorized to form image feature data of each image.

[0071] It should be noted that the features of the image feature data after vectorization are used for subsequent similarity analysis and clustering.

[0072] According to an embodiment of the present invention, the method of obtaining multiple preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, performing image segmentation processing on each image in the endoscopic image set based on the similarity, and obtaining a second image set is specifically as follows:

[0073] Acquire multiple preset lesion image features from the system database;

[0074] Marking a preset lesion image feature, and performing feature vectorization on the preset lesion image feature to form a contrast feature;

[0075] In the endoscope image set, each image is divided into multiple image regions, feature vectors corresponding to the multiple image regions are obtained through image feature data of each image, similarity analysis is performed on the feature vectors and the contrast features based on the standard Euclidean distance, and image regions with high similarity are screened out based on a first preset similarity;

[0076] The image regions with high similarity of each image are segmented and extracted, and the extracted images are integrated to form a second image set.

[0077] It should be noted that, among the multiple preset lesion image features, each preset lesion image feature corresponds to an endoscopic image feature of a disease, and the feature is used for comparative analysis.

[0078] The dividing of each image into a plurality of image regions is specifically performed by grid division, and the number of divisions is set by the user or the system. The preset similarity threshold and the first preset similarity are both set by the user.

[0079] According to an embodiment of the present invention, the image feature data in the second image set is used as a clustering sample, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result, which is specifically:

[0080] Using image feature data of each image in the second image set as clustering sample data;

[0081] Based on the DBSCAN clustering model, cluster sample data are clustered. The radiation radius of sample points is set before clustering. During the clustering process, the similarity between sample data is calculated through the standard Euclidean distance and the clustering results are obtained.

[0082] The clustering result includes a plurality of clustering groups divided from the second image set.

[0083] It should be noted that the image feature data of each image in the second image set is specifically obtained by obtaining the image features of the corresponding area in the image feature data of the endoscopic image set based on the segmentation result. The image feature data of the endoscopic image set includes the feature data of the entire image. The clustering result is used to store the partitioning information, and the corresponding multiple clustering groups can be obtained through the clustering result. Among the multiple clustering groups, each clustering group includes at least one image.

[0084] According to an embodiment of the present invention, the second image set is reordered according to the clustering result, a plurality of cluster groups are obtained based on the clustering result, image features in the cluster groups are subjected to feature fusion based on DCA to form fused feature data, the fused feature data is subjected to feature similarity analysis with a preset lesion image feature, and a target cluster group is screened out based on a preset similarity threshold, specifically:

[0085] Obtain multiple clustering groups through clustering results;

[0086] The clustering group sequence is set, and the second image set is reordered based on the clustering group sequence. After the reordering, the second image set includes a plurality of image sets, and each image set corresponds to all images in a clustering group.

[0087] Taking a cluster group as an analysis unit, the image feature data of each image in the cluster group is subjected to feature fusion based on DCA. During the fusion process, a corresponding feature map is first generated based on the image feature data of each image, and the feature map is input into a preset neural network for weighted fusion, and fused feature data is generated;

[0088] Performing feature vectorization and feature similarity calculation on the fused feature data and the one of the preset lesion image features, the similarity calculation is based on the standard Euclidean distance method, and the similarity calculation result is compared with the preset similarity threshold. If it is higher than the preset similarity threshold, it is marked as a target cluster group;

[0089] Multiple cluster groups are judged and all target cluster groups are screened out.

[0090] It should be noted that after re-sorting, images with similar lesion features can be classified and sorted, thereby improving the effect of auxiliary decision-making based on subsequent lesion analysis. The setting of the cluster group order can be based on a random order. The preset neural network is a convolutional neural network. DCA is a Dynamic Class Activity feature fusion algorithm.

[0091] According to an embodiment of the present invention, the feature enhancement of the images in the target cluster group based on histogram equalization and the image integration of the images in the target cluster group and the non-target cluster group to form a recommended image set are specifically as follows:

[0092] The image histogram and cumulative distribution function are calculated for each image of the target cluster group, and each image is enhanced by histogram equalization to obtain an enhanced image;

[0093] Based on the position of each image of the target cluster group in the second image set, performing image replacement on the enhanced image to form an enhanced second image set;

[0094] In the enhanced second image set, images in the target cluster group and the non-target cluster group are annotated to form a recommended image set;

[0095] The recommended image set is sent to a preset display terminal.

[0096] It should be noted that the second image set includes a target cluster group and a non-target cluster group. In the annotation of the images in the target cluster group and the non-target cluster group, the images in the target cluster group are images with certain image enhancement processing, which is convenient for users to quickly analyze and classify, and the image set of the target cluster group is the set of images that is closest to the preset disease characteristics.

[0097] Based on the multiple preset lesion image features, multiple recommended image sets are generated, each preset lesion image feature generates a corresponding recommended image set, and the analysis process is consistent, and the analysis is performed through the same endoscopic image set. Among the multiple preset lesion image features, the features are obtained by extracting features from historical images or manually marked lesion images as real-time analysis and comparison features.

[0098] In traditional technology, the features of endoscopic images are generally difficult to distinguish, and they need to rely on manual experience for analysis and lesion annotation. The process relies heavily on manual experience, and the existing technology lacks an effective sorting and classification process for endoscopic images, and the information integration capability is insufficient. To solve the above problems, the present invention forms a segmented image set by performing disease similarity analysis and effective segmentation on endoscopic images. This process can effectively extract useful features for manual screening and analysis, and then cluster and group the image set, reorder the image set according to the grouping results to form a recommended image set. The present invention generates a corresponding recommended image set based on each disease for medical professionals to perform reasonable analysis. In the recommended image set, high-feature and low-feature images are effectively reordered, and images based on similar features are orderly combined and arranged, effectively assisting doctors to make quick and accurate diagnoses, and based on user needs, different recommended image sets can be generated for different diseases to assist doctors in quickly analyzing disease images for different diseases, greatly improving the efficiency of endoscopic analysis.

[0099] According to an embodiment of the present invention, it also includes:

[0100] Acquire a preset lesion image feature;

[0101] Acquire a sample image of the preset lesion image feature from a system database;

[0102] Standardize the sample image and extract features based on shape, contour and color to obtain sample feature data;

[0103] Constructing a generative network based on GAN, wherein the generative network includes a generator, a discriminator and a preset loss function;

[0104] The sample feature data is imported into the generation network as real data, and the generator learns the features of the real data and generates simulated feature data;

[0105] Perform data judgment on the simulated feature data based on the discriminator, and optimize the generator and discriminator through the judgment results and the preset loss function;

[0106] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0107] Generate a preset amount of simulated images based on the trained generator;

[0108] Calculate the image histogram and cumulative distribution function of the simulated image, and perform image enhancement on the simulated image through histogram equalization to obtain an enhanced simulated image;

[0109] The enhanced simulated image is subjected to feature extraction to form a second preset lesion image feature.

[0110] It should be noted that the second preset lesion image feature is a supplementary data of the preset lesion image feature, which can be directly used as comparison data. In the image recognition task of endoscopic images, due to the limited amount of feature data in the database, there is often a situation where the amount of comparison feature data for a certain disease is small. When performing image analysis and recognition, the recognition rate of the disease is low and the feature extraction and analysis effect is poor. Therefore, the present invention obtains the original feature image, generates simulated features based on the GAN model, and finally improves the image features of the simulated data through image enhancement, and imports the simulated data as supplementary data into the corresponding database, so as to realize efficient endoscopic image feature recognition and analysis with a small amount of comparison sample data, improve the efficiency of diagnostic decision-making, and effectively improve the recognition and analysis efficiency of multiple diseases.

[0111] Figure 2 A block diagram of an endoscopic image processing system based on model analysis of the present invention is shown.

[0112] The second aspect of the present invention further provides an endoscopic image processing system 2 based on model analysis, the system comprising: a memory 21, a processor 22, the memory comprising an endoscopic image processing program based on model analysis, the endoscopic image processing program based on model analysis being executed by the processor to implement the following steps:

[0113] Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set;

[0114] Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data;

[0115] Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set;

[0116] The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result;

[0117] Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold;

[0118] Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group;

[0119] Based on multiple preset lesion image features, multiple recommended image sets are generated.

[0120] It should be noted that the target users are generally patients.

[0121] According to an embodiment of the present invention, the endoscopic video stream data of the target user is obtained, and key frames are extracted from the video stream data to obtain an endoscopic image set, specifically:

[0122] Obtain the endoscope video stream data of the target user;

[0123] The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

[0124] It should be noted that the endoscopic image set includes multiple images, which are generally stored in time sequence or in disorder.

[0125] According to an embodiment of the present invention, the feature extraction based on shape, contour and color is performed on each image in the endoscopic image set to obtain image feature data, specifically:

[0126] The endoscopic image set was preprocessed for image noise reduction and standardization;

[0127] Based on the three dimensions of shape, contour and color, feature extraction is performed on each image in the endoscope image set. The shape and contour dimensions are based on edge detection operators, and the color dimension is based on color moment calculation to obtain feature data of the three dimensions.

[0128] The feature data of the three dimensions are vectorized to form image feature data of each image.

[0129] It should be noted that the features of the image feature data after vectorization are used for subsequent similarity analysis and clustering.

[0130] According to an embodiment of the present invention, the method of obtaining multiple preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, performing image segmentation processing on each image in the endoscopic image set based on the similarity, and obtaining a second image set is specifically as follows:

[0131] Acquire multiple preset lesion image features from the system database;

[0132] Marking a preset lesion image feature, and performing feature vectorization on the preset lesion image feature to form a contrast feature;

[0133] In the endoscope image set, each image is divided into multiple image regions, feature vectors corresponding to the multiple image regions are obtained through image feature data of each image, similarity analysis is performed on the feature vectors and the contrast features based on the standard Euclidean distance, and image regions with high similarity are screened out based on a first preset similarity;

[0134] The image regions with high similarity of each image are segmented and extracted, and the extracted images are integrated to form a second image set.

[0135] It should be noted that, among the multiple preset lesion image features, each preset lesion image feature corresponds to an endoscopic image feature of a disease, and the feature is used for comparative analysis.

[0136] The dividing of each image into a plurality of image regions is specifically performed by grid division, and the number of divisions is set by the user or the system. The preset similarity threshold and the first preset similarity are both set by the user.

[0137] According to an embodiment of the present invention, the image feature data in the second image set is used as a clustering sample, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result, which is specifically:

[0138] Using image feature data of each image in the second image set as clustering sample data;

[0139] Based on the DBSCAN clustering model, cluster sample data are clustered. The radiation radius of sample points is set before clustering. During the clustering process, the similarity between sample data is calculated through the standard Euclidean distance and the clustering results are obtained.

[0140] The clustering result includes a plurality of clustering groups divided from the second image set.

[0141] It should be noted that the image feature data of each image in the second image set is specifically obtained by obtaining the image features of the corresponding area in the image feature data of the endoscopic image set based on the segmentation result. The image feature data of the endoscopic image set includes the feature data of the entire image. The clustering result is used to store the partitioning information, and the corresponding multiple clustering groups can be obtained through the clustering result. Among the multiple clustering groups, each clustering group includes at least one image.

[0142] According to an embodiment of the present invention, the second image set is reordered according to the clustering result, a plurality of cluster groups are obtained based on the clustering result, image features in the cluster groups are subjected to feature fusion based on DCA to form fused feature data, the fused feature data is subjected to feature similarity analysis with a preset lesion image feature, and a target cluster group is screened out based on a preset similarity threshold, specifically:

[0143] Obtain multiple clustering groups through clustering results;

[0144] The clustering group sequence is set, and the second image set is reordered based on the clustering group sequence. After the reordering, the second image set includes a plurality of image sets, and each image set corresponds to all images in a clustering group.

[0145] Taking a cluster group as an analysis unit, the image feature data of each image in the cluster group is subjected to feature fusion based on DCA. During the fusion process, a corresponding feature map is first generated based on the image feature data of each image, and the feature map is input into a preset neural network for weighted fusion, and fused feature data is generated;

[0146] Performing feature vectorization and feature similarity calculation on the fused feature data and the one of the preset lesion image features, the similarity calculation is based on the standard Euclidean distance method, and the similarity calculation result is compared with the preset similarity threshold. If it is higher than the preset similarity threshold, it is marked as a target cluster group;

[0147] Multiple cluster groups are judged and all target cluster groups are screened out.

[0148] It should be noted that after re-sorting, images with similar lesion features can be classified and sorted, thereby improving the effect of auxiliary decision-making based on subsequent lesion analysis. The setting of the cluster group order can be based on a random order. The preset neural network is a convolutional neural network. DCA is a Dynamic Class Activity feature fusion algorithm.

[0149] According to an embodiment of the present invention, the feature enhancement of the images in the target cluster group based on histogram equalization and the image integration of the images in the target cluster group and the non-target cluster group to form a recommended image set are specifically as follows:

[0150] The image histogram and cumulative distribution function are calculated for each image of the target cluster group, and each image is enhanced by histogram equalization to obtain an enhanced image;

[0151] Based on the position of each image of the target cluster group in the second image set, performing image replacement on the enhanced image to form an enhanced second image set;

[0152] In the enhanced second image set, images in the target cluster group and the non-target cluster group are annotated to form a recommended image set;

[0153] The recommended image set is sent to a preset display terminal.

[0154] It should be noted that the second image set includes a target cluster group and a non-target cluster group. In the annotation of the images in the target cluster group and the non-target cluster group, the images in the target cluster group are images with certain image enhancement processing, which is convenient for users to quickly analyze and classify, and the image set of the target cluster group is the set of images that is closest to the preset disease characteristics.

[0155] Based on the multiple preset lesion image features, multiple recommended image sets are generated, each preset lesion image feature generates a corresponding recommended image set, and the analysis process is consistent, and the analysis is performed through the same endoscopic image set. Among the multiple preset lesion image features, the features are obtained by extracting features from historical images or manually marked lesion images as real-time analysis and comparison features.

[0156] In traditional technology, the features of endoscopic images are generally difficult to distinguish, and they need to rely on manual experience for analysis and lesion annotation. The process relies heavily on manual experience, and the existing technology lacks an effective sorting and classification process for endoscopic images, and the information integration capability is insufficient. To solve the above problems, the present invention forms a segmented image set by performing disease similarity analysis and effective segmentation on endoscopic images. This process can effectively extract useful features for manual screening and analysis, and then cluster and group the image set, reorder the image set according to the grouping results to form a recommended image set. The present invention generates a corresponding recommended image set based on each disease for medical professionals to perform reasonable analysis. In the recommended image set, high-feature and low-feature images are effectively reordered, and images based on similar features are orderly combined and arranged, effectively assisting doctors to make quick and accurate diagnoses, and based on user needs, different recommended image sets can be generated for different diseases to assist doctors in quickly analyzing disease images for different diseases, greatly improving the efficiency of endoscopic analysis.

[0157] The third aspect of the present invention also provides a computer-readable storage medium, which includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by a processor, the steps of the endoscopic image processing method based on model analysis as described in any one of the above items are implemented.

[0158] The present invention discloses an endoscopic image processing system and method based on model analysis. By acquiring endoscopic video stream data, key frames are extracted to form an image set, and then shape, contour and color features are extracted for each image. Similarity analysis is performed using preset lesion image features, and the image set is segmented and clustered to obtain multiple cluster groups. The target cluster group is screened by DCA feature fusion and similarity threshold, and histogram equalization is performed on it. Finally, the target and non-target cluster group images are integrated to generate a recommended image set, and multiple recommended sets are generated based on multiple preset lesion features to assist in diagnosis and analysis. Through the present invention, medical staff can be assisted in performing rapid endoscopic image diagnosis and analysis of different diseases, effectively improving the efficiency of endoscopic analysis. In the several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0159] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0160] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0161] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and other media that can store program codes.

[0162] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0163] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An endoscopic image processing method based on model analysis, characterized in that: include: Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set; Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data; Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set; The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result; Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold; Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group; Based on a variety of preset lesion image features, multiple recommended image sets are generated.

2. The endoscopic image processing method based on model analysis according to claim 1, characterized in that: The step of obtaining the endoscope video stream data of the target user and extracting key frames from the video stream data to obtain an endoscope image set is as follows: Obtain the endoscope video stream data of the target user; The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

3. The endoscopic image processing method based on model analysis according to claim 2, characterized in that: The feature extraction based on shape, contour and color is performed on each image in the endoscope image set to obtain image feature data, specifically: The endoscopic image set was preprocessed for image noise reduction and standardization; Based on the three dimensions of shape, contour and color, feature extraction is performed on each image in the endoscope image set. The shape and contour dimensions are based on edge detection operators, and the color dimension is based on color moment calculation to obtain feature data of the three dimensions. The feature data of the three dimensions are vectorized to form image feature data of each image.

4. The endoscopic image processing method based on model analysis according to claim 3 is characterized in that: The method of obtaining a plurality of preset lesion image features, performing feature similarity analysis on the image feature data and a preset lesion image feature, performing image segmentation processing on each image in the endoscopic image set based on the similarity, and obtaining a second image set is specifically as follows: Acquire multiple preset lesion image features from the system database; Marking a preset lesion image feature, and performing feature vectorization on the preset lesion image feature to form a contrast feature; In the endoscope image set, each image is divided into multiple image regions, feature vectors corresponding to the multiple image regions are obtained through image feature data of each image, similarity analysis is performed on the feature vectors and the contrast features based on the standard Euclidean distance, and image regions with high similarity are screened out based on a first preset similarity; The image regions with high similarity of each image are segmented and extracted, and the extracted images are integrated to form a second image set.

5. The method for endoscopic image processing based on model analysis according to claim 4, characterized in that: The image feature data in the second image set is used as a clustering sample, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result, which is specifically: Using image feature data of each image in the second image set as clustering sample data; Based on the DBSCAN clustering model, cluster sample data are clustered. The radiation radius of sample points is set before clustering. During the clustering process, the similarity between sample data is calculated through the standard Euclidean distance and the clustering results are obtained. The clustering result includes a plurality of clustering groups divided from the second image set.

6. The method for endoscopic image processing based on model analysis according to claim 5, characterized in that: The second image set is reordered according to the clustering result, a plurality of cluster groups are obtained based on the clustering result, image features in the cluster groups are subjected to feature fusion based on DCA to form fused feature data, the fused feature data is subjected to feature similarity analysis with a preset lesion image feature, and a target cluster group is screened out based on a preset similarity threshold, specifically: Obtain multiple clustering groups through clustering results; The clustering group sequence is set, and the second image set is reordered based on the clustering group sequence. After the reordering, the second image set includes a plurality of image sets, and each image set corresponds to all images in a clustering group. Taking a cluster group as an analysis unit, the image feature data of each image in the cluster group is subjected to feature fusion based on DCA. During the fusion process, a corresponding feature map is first generated based on the image feature data of each image, and the feature map is input into a preset neural network for weighted fusion, and fused feature data is generated; Performing feature vectorization and feature similarity calculation on the fused feature data and the one of the preset lesion image features, the similarity calculation is based on the standard Euclidean distance method, and the similarity calculation result is compared with the preset similarity threshold. If it is higher than the preset similarity threshold, it is marked as a target cluster group; Multiple cluster groups are judged and all target cluster groups are screened out.

7. The method for endoscopic image processing based on model analysis according to claim 6, characterized in that: The feature enhancement of the images in the target cluster group based on histogram equalization and the image integration of the images in the target cluster group and the non-target cluster group to form a recommended image set are specifically as follows: The image histogram and cumulative distribution function are calculated for each image of the target cluster group, and each image is enhanced by histogram equalization to obtain an enhanced image; Based on the position of each image of the target cluster group in the second image set, performing image replacement on the enhanced image to form an enhanced second image set; In the enhanced second image set, images in the target cluster group and the non-target cluster group are annotated to form a recommended image set; The recommended image set is sent to a preset display terminal.

8. An endoscopic image processing system based on model analysis, characterized in that: The system includes: a memory and a processor, wherein the memory includes an endoscopic image processing program based on model analysis, and when the endoscopic image processing program based on model analysis is executed by the processor, the following steps are implemented: Acquire endoscope video stream data of a target user, extract key frames from the video stream data, and obtain an endoscope image set; Perform feature extraction based on shape, contour, and color on each image in the endoscope image set to obtain image feature data; Acquire multiple preset lesion image features, perform feature similarity analysis on the image feature data and a preset lesion image feature, perform image segmentation processing on each image in the endoscope image set based on the similarity, and obtain a second image set; The image feature data in the second image set is used as clustering samples, and each image in the second image set is clustered based on the DBSCAN clustering model to obtain a clustering result; Reordering the second image set according to the clustering result, obtaining multiple clustering groups based on the clustering result, performing feature fusion based on DCA on the image features in the clustering group to form fused feature data, performing feature similarity analysis on the fused feature data and a preset lesion image feature, and screening out a target clustering group based on a preset similarity threshold; Based on histogram equalization, the images in the target cluster group are enhanced in features, and the recommended image set is formed based on the image integration of the target cluster group and the images in the non-target cluster group; Based on a variety of preset lesion image features, multiple recommended image sets are generated.

9. The endoscopic image processing system based on model analysis according to claim 8, characterized in that: The step of obtaining the endoscope video stream data of the target user and extracting key frames from the video stream data to obtain an endoscope image set is as follows: Obtain the endoscope video stream data of the target user; The key frames of the video stream data are extracted and the corresponding time information is recorded to form an endoscopic image set.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an endoscopic image processing program based on model analysis. When the endoscopic image processing program based on model analysis is executed by a processor, the steps of the endoscopic image processing method based on model analysis as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Image processing method and system for glaucoma analysis and medium

    CN118247267A

  • LED energy-saving control method and system based on Internet of Things, and medium

    CN118301821A

  • Image processing device, endoscope system, and image processing method

    US20170083791A1

Cited By

  • Children intestinal tract lesion area detection method based on machine vision

    CN119600007A