Otoscope image processing system for establishing database model analysis based on deep learning

Abstract

The invention provides an otoscope image processing system for establishing database model analysis based on deep learning, and relates to the technical field of image processing, the otoscope image processing system comprises an acquisition layer, a processing layer, an extraction layer, a modeling layer and an analysis layer, the processing layer adopts an AI image expansion algorithm to carry out fusion splicing and interference elimination processing on acquired images, and the extraction layer adopts a mixed feature extraction method to extract color, texture and shape features of the images processed by the processing layer and introduce MPEG-7 descriptors to express the features; according to the method, otoscope images of a patient are collected, features and feature values of the images are extracted, a focus global shared image library is constructed through deep learning modeling, the focus global shared image library is compared with the otoscope images of the patient for training, and corresponding focuses are determined, so that different ear diseases can be automatically identified and diagnosed by learning and analyzing a large amount of ear image data, and the diagnosis accuracy is improved. And more accurate and timely diagnosis information is provided for doctors.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an otoscope image processing system for establishing a database model analysis based on deep learning. Background Art

[0002] An otoscope, also known as an ear speculum or ear endoscope, is a non-invasive medical device that inserts a tiny probe into the external auditory canal and uses optical magnification principles or electronic imaging technology to clearly and intuitively observe the health of the external auditory canal, tympanic membrane, and some middle ear structures. It is used to diagnose ear canal diseases, tympanic membrane perforation, middle ear inflammation, and other problems. It is an indispensable tool in otolaryngology examinations.

[0003] Otoscope examination generally combines manual observation and modern technical means. Otoscopes, including traditional ear specula, electric otoscopes and more advanced video otoscopes, are commonly used examination tools in otolaryngology. During the examination, doctors usually use otoscopes to observe the patient's external auditory canal, eardrum and part of the middle ear structure to determine whether there is disease or abnormality. For traditional ear specula, doctors need to rely on their own experience and naked eyes to observe the ear structure, which relies on manual identification to a certain extent.

[0004] With the development of science and technology, modern equipment such as electric otoscopes and video otoscopes have become increasingly popular. They not only provide clear and magnified images, but also make the detection process more intuitive and accurate. When using an electric otoscope or a video otoscope for detection, doctors can observe the ear structure through the image on the device screen. This image is usually electronically magnified and processed to show details more clearly. However, doctors still need to rely on their own professional knowledge and experience to judge abnormalities in the image. Therefore, the present invention proposes an otoscope image processing system based on deep learning to establish a database model analysis to solve the problems existing in the prior art. Summary of the invention

[0005] In response to the above problems, the present invention proposes an otoscope image processing system based on deep learning to establish a database model analysis. The otoscope image processing system based on deep learning to establish a database model analysis can automatically identify and diagnose different ear diseases by learning and analyzing a large amount of ear image data, thereby providing doctors with more accurate and timely diagnostic information.

[0006] To achieve the purpose of the present invention, the present invention is implemented through the following technical solutions: an otoscope image processing system for database model analysis based on deep learning, including an acquisition layer, a processing layer, an extraction layer, a modeling layer and an analysis layer, wherein the acquisition layer is used to connect to an acquisition device to acquire patient ear images, the processing layer adopts an AI image expansion algorithm to fuse and splice the acquired images, and eliminate interference, the extraction layer adopts a hybrid feature extraction method to extract the color, texture, and shape features of the image processed by the processing layer, and introduces MPEG-7 descriptors to represent the above features to construct a sample;

[0007] The modeling layer is used to collect and organize representative images of various types of ear lesions, build a global shared image library of lesions, build a deep neural network model based on the image library using a deep learning framework, and automatically learn the feature representation in the image library in combination with a hybrid feature extraction method. The analysis layer compares and trains the samples and the deep neural network model, calculates the feature similarity between each other, determines the lesion type and lesion progression based on the similarity sorting results, and outputs the lesion recognition results.

[0008] A further improvement is that the acquisition layer includes an acquisition unit, which is used to connect to a digital otoscope device to acquire images of the patient's ear, and the acquisition unit provides a human-computer interaction channel for setting the acquisition resolution, exposure and contrast, thereby controlling the parameters of the acquired image.

[0009] A further improvement is that the acquisition layer also includes a compatible unit, which uses a Bt1120 interface to simulate RGB and HDMI input interfaces, is compatible with image inputs of different types of digital otoscope devices, stores them in a storage cloud disk, and then uses VPSS to scale and synthesize the input images and output them to the processing layer.

[0010] The further improvement is that: the processing layer includes a preprocessing module, which is used to analyze the parameter reference values ​​of the image, including parameters of image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green, and blue saturation. Then, it is determined whether the above analysis data meets the set requirements. If not, the VCA algorithm is used to extract the noisy endmember spectrum in the video image, and the extracted endmember spectrum is denoised by the SSA algorithm. The endmember extraction algorithm is used to extract the endmember spectrum Y, and its trajectory matrix is ​​calculated, and singular value decomposition, grouping and reconstruction are performed. Then, the super-resolution algorithm SRCNN is used, based on the convolutional neural network, with the original video image as input, and the resolution is increased to the set value by the upsampling algorithm. Then, after 3 layers of convolution operations of 9*9*128, 3*3*64, and 5*5 respectively, the super-resolution output is obtained to enhance the video image resolution. Finally, the RIFT real-time video interpolation algorithm is used to interpolate the video image and adjust the refresh rate.

[0011] Further improvements are: the processing layer also includes an AI image expansion module, which is based on an image super-resolution algorithm based on deep learning. It trains a neural image expansion model according to the ear images in the medical database, and uses the image expansion model to analyze and reason the image data processed by the preprocessing module, overlap and compare several scattered images with the complete image in the model, determine the corresponding positions of the scattered images in the ear, sort and fuse several scattered images according to the expanded complete image according to position reasoning, and combine them with other complete images to generate a comprehensive image.

[0012] A further improvement is that the extraction layer includes a feature extraction module, which is used to access the comprehensive image provided by the processing layer to perform feature extraction, including the following steps:

[0013] Based on clinical and medical annotations, define the tympanic membrane features CMEF;

[0014] Using image segmentation and edge detection, the defined CMEF features are automatically extracted from the image;

[0015] Use color histogram and color moment to extract the distribution and statistical characteristics of colors in the image;

[0016] LBP local binary pattern and gray level co-occurrence matrix are used to extract the texture features of the image;

[0017] Use contour detection and Hough transform to extract shape features of images;

[0018] Descriptors introduced in the MPEG-7 standard describe the color, texture, and shape characteristics of an image.

[0019] A further improvement is that the extraction layer also includes a merging set module, which is used to fuse CMEF features, color features, texture features, shape features and MPEG-7 descriptor high-level features to form a comprehensive feature vector and construct a sample.

[0020] Further improvements are as follows: the modeling layer includes a database unit, which is used to collect and organize representative images of various types of ear lesions, build a global shared image library of lesions, annotate each image in the image library, clarify the type, progression, and location information of the lesions, and the database unit provides a manual channel for maintenance personnel to regularly update the image library and incorporate new cases and research results.

[0021] Further improvements are as follows: the modeling layer also includes a modeling unit, which uses the CNN deep learning framework, combined with a hybrid feature extraction method, to learn feature representation from a global shared image library of lesions, captures features in the image through multi-layer convolution and pooling operations, uses transfer learning technology to accelerate the training process, and introduces an integrated learning classifier - fuzzy stacked generalized FSG, fuses the decisions of multiple base classifiers, and constructs a deep neural network model.

[0022] Further improvements are as follows: the analysis layer includes a comparison training module and a report output module, the comparison training module is used to access the extraction layer and the modeling layer, represent and compare the features of the extracted samples with the features in the global shared image library of lesions, calculate the distance or similarity index between the feature vectors, sort the candidate lesion types according to the calculated similarity index, select the lesion type with the highest similarity as the recognition result according to the similarity sorting result, and then repeat the above comparison process to sort and identify the course of the lesion, and determine the course of the lesion and the specific location; the report output module is used to present the results of the analysis in a visual form, and mark the corresponding lesion type label in the lesion area on the original image, synchronously access the medical database, and retrieve the quality plan for dealing with the lesion.

[0023] The beneficial effects of the present invention are:

[0024] 1. The present invention collects patients' otoscope images, extracts image features and eigenvalues, constructs a global shared image library of lesions through deep learning modeling, and compares and trains with patients' otoscope images to determine corresponding lesions. Thus, by learning and analyzing a large amount of ear image data, different ear diseases can be automatically identified and diagnosed, providing doctors with more accurate and timely diagnostic information.

[0025] 2. During diagnosis, the present invention uses AI image expansion to analyze and reason about possible scattered images, overlaps and compares several scattered images with the complete image, and determines the corresponding positions of the scattered images in the ear. In this way, the image expansion and fusion are carried out to ensure the completeness of the image analysis and facilitate the determination of the locations of a series of lesions.

[0026] 3. In the comparative training, the present invention calculates the distance or similarity index between the feature vectors, sorts the candidate lesion types, selects the lesion type with the highest similarity as the recognition result according to the similarity sorting result, and then repeats the process to sort and identify the course of the lesion, determine the course of the lesion, and further improve the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the structural composition of the present invention. DETAILED DESCRIPTION

[0028] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0029] Embodiment 1

[0030] according to Figure 1 As shown, this embodiment proposes an otoscope image processing system for establishing a database model analysis based on deep learning, including an acquisition layer, a processing layer, an extraction layer, a modeling layer and an analysis layer. The acquisition layer is used to connect to the acquisition device to acquire the patient's ear image. The processing layer adopts an AI image expansion algorithm to fuse and splice the acquired images, and eliminate interference. The extraction layer adopts a hybrid feature extraction method to extract the color, texture, and shape features of the image processed by the processing layer, and introduces MPEG-7 descriptors to represent the above features to construct a sample.

[0031] The modeling layer is used to collect and organize representative images of various types of ear lesions, build a global shared image library of lesions, build a deep neural network model based on the image library using a deep learning framework, and automatically learn the feature representation in the image library by combining a hybrid feature extraction method. The analysis layer compares and trains the samples and the deep neural network model, calculates the feature similarity between each other, and determines the lesion type and lesion process according to the similarity sorting result, and outputs the lesion recognition result. When in use, collect the patient's otoscope image, extract the image's features and feature values, build a global shared image library of lesions through deep learning modeling, and compare and train with the patient's otoscope image to determine the corresponding lesion. Thus, by learning and analyzing a large amount of ear image data, different ear diseases can be automatically identified and diagnosed, providing doctors with more accurate and timely diagnostic information.

[0032] Embodiment 2

[0033] according to Figure 1 As shown, this embodiment proposes an otoscope image processing system for establishing a database model analysis based on deep learning, including an acquisition layer, a processing layer, an extraction layer, a modeling layer and an analysis layer. The acquisition layer is used to connect to the acquisition device to acquire the patient's ear image. The processing layer adopts an AI image expansion algorithm to fuse and splice the acquired images, and eliminate interference. The extraction layer adopts a hybrid feature extraction method to extract the color, texture, and shape features of the image processed by the processing layer, and introduces MPEG-7 descriptors to represent the above features to construct a sample.

[0034] The modeling layer is used to collect and organize representative images of various types of ear lesions, build a global shared image library of lesions, build a deep neural network model based on the image library using a deep learning framework, and automatically learn the feature representation in the image library in combination with a hybrid feature extraction method. The analysis layer compares and trains the samples and the deep neural network model, calculates the feature similarity between each other, determines the lesion type and lesion progression based on the similarity sorting results, and outputs the lesion recognition results.

[0035] The acquisition layer includes an acquisition unit and a compatible unit. The acquisition unit is used to connect to a digital otoscope device to acquire images of the patient's ears, and the acquisition unit provides a human-computer interaction channel for setting the acquisition resolution, exposure and contrast, thereby controlling the parameters of the acquired image. The compatible unit uses a Bt1120 interface to simulate RGB and HDMI input interfaces, is compatible with image inputs of different types of digital otoscope devices, and stores them in a storage cloud disk. Then, VPSS is used to scale and synthesize the input images and output them to the processing layer.

[0036] The processing layer includes a preprocessing module and an AI image expansion module. The preprocessing module is used to analyze the parameter reference values ​​of the image, including image resolution, image refresh rate, image noise, image white balance, image brightness, image red, green, and blue saturation parameters. Then, it is determined whether the above analysis data meets the set requirements. If not, the VCA algorithm is used to extract the noisy end member spectrum in the video image, and the extracted end member spectrum is denoised by the SSA algorithm. The end member extraction algorithm is used to extract the end member spectrum Y, and its trajectory matrix is ​​calculated, and singular value decomposition, grouping and reconstruction are performed. Then, the super-resolution algorithm SRCNN is used. Based on the convolutional neural network, the original video image is used as input, and the resolution is increased to the set value by the upsampling algorithm. Then, after 3 layers of convolution operations of 9*9*128, 3*3*64, and 5*5, the super-resolution output is obtained to enhance the video image resolution. Finally, the RIFT real-time video interpolation algorithm is used to interpolate the video image and adjust the refresh rate. The network structure of SRCNN contains three levels: through a layer of CNN and ReLU activation function, the features of the low-resolution image are extracted and stored in multiple feature maps. The feature map obtained by the feature extraction layer is further processed by the convolution kernel and ReLU layer for nonlinear mapping to deepen the network depth and better learn the mapping relationship from low resolution to high resolution. The feature map output by the nonlinear mapping layer is restored to a high-resolution image through the convolution operation, and the loss is calculated with the real high-resolution image (Ground-Truth), and the parameters of the entire network are optimized through the back-propagation algorithm.

[0037] SRCNN uses deep learning technology, especially convolutional neural network (CNN), to achieve image super-resolution (SR), including the following methods:

[0038] 1. Preprocessing: video frame extraction and upsampling:

[0039] First, a single frame image is extracted from the video as input. Since the video is composed of a series of continuous images (frames), the video file is decomposed into individual image frames.

[0040] Then, each extracted frame image is preliminarily upsampled to improve processing efficiency or adapt to specific network input requirements, and the low-resolution image (LR) is enlarged to an approximate value of the target resolution through a simple interpolation method (such as bilinear interpolation or bicubic interpolation).

[0041] 2. SRCNN network processing:

[0042] The SRCNN network structure consists of three convolutional layers, and the specific configuration of each layer is as follows:

[0043] The first convolutional layer (9*9*128):

[0044] Input: preprocessed image (if preliminary upsampling is performed, it is the upsampled image; otherwise it is the original low-resolution image).

[0045] Operation: Use a 9*9 convolution kernel for convolution operation, and the depth of the output feature map (that is, the number of convolution kernels) is 128. This layer is used to extract deep features of the image.

[0046] Activation function: Use ReLU (Rectified Linear Unit) activation function to increase the nonlinear ability of the network.

[0047] The second convolutional layer (3*3*64):

[0048] Input: The output of the first convolutional layer.

[0049] Operation: Use a 3*3 convolution kernel to further process the feature map, and the depth of the output feature map is reduced to 64. This layer continues to extract and abstract image features.

[0050] Activation function: ReLU activation function is also used.

[0051] The third convolutional layer (5*5):

[0052] Input: The output of the second convolutional layer.

[0053] Operation: A 5*5 convolution kernel is used to generate the final high-resolution image (HR). The number of output channels of this layer matches the number of color channels of the target image (for example, for RGB images, the number of output channels is 3).

[0054] Activation Function: This layer does not use a non-linear activation function since the goal is to generate a pixel-wise output image.

[0055] 3. Post-processing and video reconstruction:

[0056] The high-resolution image frames output by the SRCNN network are reassembled into a video.

[0057] The combined video is further post-processed, such as denoising and sharpening, to further improve the visual effect.

[0058] 4. Evaluation and Tuning:

[0059] Evaluation metrics such as PSNR, SSIM, etc. are used to assess the quality of super-resolution videos.

[0060] Adjust network parameters or optimize algorithms based on the evaluation results to further improve the super-resolution effect.

[0061] Through the above steps, the SRCNN algorithm converts low-resolution video images into high-resolution images, thereby significantly improving the clarity and visual effects of the video.

[0062] The AI ​​image expansion module is based on an image super-resolution algorithm of deep learning. It trains a neural image expansion model according to the ear images in the medical database. The image expansion model is used to analyze and infer the image data processed by the preprocessing module, and several scattered images are overlapped and compared with the complete image in the model to determine the corresponding positions of the scattered images in the ear. According to positional reasoning, the several scattered images are sorted and fused according to the expanded complete image, and combined with other complete images to generate a comprehensive image. The PyTorch Geometic neural network and the deep learning library built based on PyTorch are used to model and train unstructured data, and high-performance training is achieved using dedicated CUDA kernels. The PyTorch-based graph neural network framework supports multiple graph neural network models such as graph convolutional network (GCN), graph attention network (GAT), and variant of graph convolutional network (GraphSAGE), and provides graph data sets and preprocessors. The present invention uses AI to expand the image to improve the quality and accuracy of the image. Through AI expansion, the feature extraction ability of the neural network can be utilized to retain more details in the original image, making the enlarged image more realistic and clear. AI expansion is used to automatically learn the structure and texture features of the image through training data, so that several scattered images are overlapped and compared with the complete image in the model to determine the corresponding positions of the scattered images in the ear.

[0063] The extraction layer includes a feature extraction module and a merging set module. The feature extraction module is used to access the comprehensive image provided by the processing layer to perform feature extraction, including the following steps:

[0064] Based on clinical and medical annotations, define tympanic membrane features CMEF; based on clinical experience and medical knowledge, define a series of clinically significant tympanic membrane features (CMEF), such as the position of the light cone, visibility of the malleus, protrusion of the membrane, presence of a tympanostomy tube or the presence of wax.

[0065] Using image segmentation and edge detection, the defined CMEF features are automatically extracted from the image; using image processing techniques, such as image segmentation and edge detection, these CMEF features are automatically or semi-automatically extracted from the pre-processed otoscope image.

[0066] Color histogram and color moment are used to extract the distribution and statistical characteristics of colors in images.

[0067] LBP local binary pattern and gray level co-occurrence matrix are used to extract the texture features of the image; the spatial relationship between pixels or pixel regions in the image is described.

[0068] Use contour detection and Hough transform to extract shape features of images, such as perimeter, area, eccentricity, etc., to describe the geometric properties of objects in the image.

[0069] Descriptors in the MPEG-7 standard are introduced to describe the color, texture and shape characteristics of images, such as the Color Layout Descriptor (CLD) and the Edge Histogram Descriptor (EHD), to more comprehensively describe the color, texture and shape characteristics of images.

[0070] The merging set module is used to fuse CMEF features, color features, texture features, shape features and MPEG-7 descriptor high-level features to form a comprehensive feature vector and construct a sample.

[0071] The modeling layer includes a database unit and a modeling unit. The database unit is used to collect and organize representative images of various ear lesions, build a global shared image library of lesions, annotate each image in the image library, and clarify the type, process, and location information of the lesions. The database unit provides a manual channel for maintenance personnel to regularly update the image library, incorporate new cases and research results, and improve the data completeness of the image library.

[0072] The modeling unit uses the CNN deep learning framework, combined with a hybrid feature extraction method, to learn feature representation from a global shared image library of lesions, captures features in the image through multi-layer convolution and pooling operations, uses transfer learning technology to accelerate the training process, and introduces an integrated learning classifier - fuzzy stacked generalization FSG, which fuses the decisions of multiple base classifiers, builds a deep neural network model, improves the accuracy and robustness of lesion identification, and continuously optimizes the structure and parameters of the deep learning model.

[0073] The analysis layer includes a contrast training module and a report output module. The contrast training module is used to access the extraction layer and the modeling layer, and to represent and compare the features of the extracted samples with the features in the global shared image library of the lesion, calculate the distance or similarity index between the feature vectors, sort the candidate lesion types according to the calculated similarity index, select the lesion type with the highest similarity as the recognition result according to the similarity sorting result, and then repeat the above-mentioned contrast process to sort and identify the course of the lesion, and determine the course of the lesion and the specific location; the report output module is used to present the results of the analysis in a visual form, and mark the corresponding lesion type label on the lesion area on the original image, synchronously access the medical database, and retrieve the quality plan for the lesion. In the contrast training, the distance or similarity index between the feature vectors is calculated, and the candidate lesion types are sorted. According to the similarity sorting result, the lesion type with the highest similarity is selected as the recognition result, and then the process is repeated to sort and identify the course of the lesion, and determine the course of the lesion, so as to further improve the accuracy of diagnosis.

[0074] The otoscope image processing system based on deep learning database model analysis collects patient otoscope images, extracts image features and eigenvalues, builds a global shared image library of lesions through deep learning modeling, and compares and trains with patient otoscope images to determine the corresponding lesions. Thus, by learning and analyzing a large amount of ear image data, it can automatically identify and diagnose different ear diseases, and provide doctors with more accurate and timely diagnostic information. At the same time, when diagnosing, the present invention uses AI to expand the image to analyze and reason the image for possible scattered images, overlaps and compares several scattered images with the complete image, and determines the corresponding position of the scattered images in the ear, so as to expand and fuse the image, which is convenient for ensuring the perfection of image analysis and convenient for determining the positions of a series of lesions. At the same time, in the comparative training, the present invention calculates the distance or similarity index between the feature vectors, sorts the candidate lesion types, selects the lesion type with the highest similarity as the recognition result according to the similarity sorting result, and then repeats the process to sort and identify the course of the lesion, and determine the course of the lesion, so as to further improve the accuracy of diagnosis.

[0075] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An otoscope image processing system for database model analysis based on deep learning, including an acquisition layer, a processing layer, an extraction layer, a modeling layer and an analysis layer, characterized in that: The acquisition layer is used to connect to the acquisition device to acquire the patient's ear image. The processing layer adopts the AI ​​image expansion algorithm to fuse and splice the acquired images, and eliminate interference. The extraction layer adopts a hybrid feature extraction method to extract the color, texture, and shape features of the image processed by the processing layer, and introduces MPEG-7 descriptors to represent the above features to construct samples. The modeling layer is used to collect and organize representative images of various types of ear lesions, build a global shared image library of lesions, build a deep neural network model based on the image library using a deep learning framework, and automatically learn the feature representation in the image library in combination with a hybrid feature extraction method. The analysis layer compares and trains the samples and the deep neural network model, calculates the feature similarity between each other, determines the lesion type and lesion progression based on the similarity sorting results, and outputs the lesion recognition results.

2. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 1, characterized in that: The acquisition layer includes an acquisition unit, which is used to connect to a digital otoscope device to acquire images of the patient's ear. The acquisition unit provides a human-computer interaction channel for setting the acquisition resolution, exposure and contrast, thereby controlling the parameters of the acquired image.

3. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 2, characterized in that: The acquisition layer also includes a compatible unit, which uses a Bt1120 interface to simulate RGB and HDMI input interfaces, is compatible with image inputs of different types of digital otoscope devices, stores them in a storage cloud disk, and then uses VPSS to scale and synthesize the input images and output them to the processing layer.

4. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 1, characterized in that: The processing layer includes a preprocessing module, which is used to analyze the parameter reference values ​​of the image, including parameters of image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green, and blue saturation. Then, it is determined whether the above analysis data meets the set requirements. If not, the VCA algorithm is used to extract the noisy endmember spectrum in the video image, and the extracted endmember spectrum is denoised by the SSA algorithm. The endmember extraction algorithm is used to extract the endmember spectrum Y, and its trajectory matrix is ​​calculated, and singular value decomposition, grouping and reconstruction are performed. Then, the super-resolution algorithm SRCNN is used, based on the convolutional neural network, with the original video image as input, and the resolution is increased to the set value by the upsampling algorithm to obtain the super-resolution output, enhance the video image resolution, and finally the RIFT real-time video interpolation algorithm is used to interpolate the video image and adjust the refresh rate.

5. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 4, characterized in that: The processing layer also includes an AI image expansion module, which is based on an image super-resolution algorithm of deep learning. It trains a neural image expansion model according to the ear images in the medical database, analyzes and infers the image data processed by the preprocessing module through the image expansion model, overlaps and compares several scattered images with the complete image in the model, determines the corresponding positions of the scattered images in the ear, sorts and fuses the several scattered images according to the expanded complete image according to position reasoning, and generates a comprehensive image by combining with other complete images.

6. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 5, characterized in that: The extraction layer includes a feature extraction module, which is used to access the comprehensive image provided by the processing layer to perform feature extraction, including the following steps: Based on clinical and medical annotations, define the tympanic membrane features CMEF; Using image segmentation and edge detection, the defined CMEF features are automatically extracted from the image; Use color histogram and color moment to extract the distribution and statistical characteristics of colors in the image; LBP local binary pattern and gray level co-occurrence matrix are used to extract the texture features of the image; Use contour detection and Hough transform to extract shape features of images; Descriptors introduced in the MPEG-7 standard describe the color, texture, and shape characteristics of an image.

7. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 6, characterized in that: The extraction layer also includes a merging set module, which is used to fuse CMEF features, color features, texture features, shape features and MPEG-7 descriptor high-level features to form a comprehensive feature vector and construct a sample.

8. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 1, characterized in that: The modeling layer includes a database unit, which is used to collect and organize representative images of various types of ear lesions, build a global shared image library of lesions, annotate each image in the image library, and clarify the type, progression, and location information of the lesions. The database unit also provides a manual channel for maintenance personnel to regularly update the image library and incorporate new cases and research results.

9. The otoscope image processing system based on deep learning to establish a database model analysis according to claim 8, characterized in that: The modeling layer also includes a modeling unit, which uses a CNN deep learning framework combined with a hybrid feature extraction method to learn feature representation from a global shared image library of lesions, captures features in the image through multi-layer convolution and pooling operations, uses transfer learning technology to accelerate the training process, and introduces an integrated learning classifier - fuzzy stacked generalization FSG, fuses the decisions of multiple base classifiers, and constructs a deep neural network model.

10. The otoscope image processing system for establishing a database model analysis based on deep learning according to any one of claims 1 to 9, characterized in that: The analysis layer includes a comparison training module and a report output module. The comparison training module is used to access the extraction layer and the modeling layer, represent and compare the features of the extracted samples with the features in the global shared image library of the lesion, calculate the distance or similarity index between the feature vectors, sort the candidate lesion types according to the calculated similarity index, select the lesion type with the highest similarity as the recognition result according to the similarity sorting result, and then repeat the above comparison process to sort and identify the course of the lesion, and determine the course of the lesion and the specific location; the report output module is used to present the analysis results in a visual form, and mark the corresponding lesion type label in the lesion area on the original image, synchronously access the medical database, and retrieve the quality plan for dealing with the lesion.

Images