An otoscope image open set quality control method

Abstract

The present application relates to otoscope image intelligent diagnosis technical field, especially to an otoscope image open set quality control method, comprising the following steps: constructing a quality control training set, the quality control training set contains positive samples and negative samples, the positive samples are effective otoscope images meeting the diagnostic criteria, and the negative samples include at least one type selected from adjacent anatomical site images, other department endoscope images and general visual interference images; using the quality control training set, adopting a lightweight convolutional neural network as a backbone network, training through a binary classification loss function to obtain a binary classification verification network; obtaining an otoscope image to be diagnosed, inputting the otoscope image to be diagnosed into the binary classification verification network, and outputting an effective probability representing that the image is an effective otoscope image; the present application adopts a lightweight convolutional neural network as a backbone, realizes extremely low false positive rate while ensuring high interception rate, and effectively protects effective images from being misblocked.

Description

Technical Field

[0001] This invention relates to the field of intelligent diagnostic technology for otoscope images, and in particular to a method for quality control of open sets of otoscope images. Background Technology

[0002] Otoscopic examination is an important means of diagnosing ear diseases. With the development of artificial intelligence technology, intelligent diagnostic systems for otoscope images based on deep learning are gradually being applied in clinical practice. However, in real clinical scenarios, due to factors such as differences in operator skills, equipment heterogeneity, and low patient cooperation, the acquired otoscope images often contain a large amount of invalid data, including: severely defocused and blurred images, extremely abnormal lighting (overexposure / underexposure), images where the tympanic membrane is not exposed in the external auditory canal, and even mistransmitted images of nasal endoscopes, laryngoscopes, or completely irrelevant images. Statistics show that such invalid images account for 10% to 15% of the actual outpatient flow.

[0003] In existing technologies, most otoscopy AI diagnostic models are based on the "closed set assumption," assuming that the input is high-quality, standardized otoscopy images. When invalid images are input, the model will forcibly output high-confidence "illusion" predictions (e.g., misclassifying a blurry black mass as "cholesteatoma of the external auditory canal"), which not only wastes computational resources but also seriously threatens medical safety. Currently, there is a lack of a lightweight quality control method that can effectively identify and intercept various invalid images in open sets.

[0004] Therefore, there is an urgent need for a method specifically for quality control of open sets of otoscope images, which can automatically determine whether the images are valid before diagnosis and intercept and prompt invalid images, thus ensuring the security and reliability of AI diagnostic systems from the source. Summary of the Invention

[0005] The purpose of this invention is to provide an open set quality control method for otoscope images, which solves the problems of lack of open set noise interception methods for otoscope images and the easy generation of hallucination predictions by the model in the prior art, and achieves efficient, lightweight and robust image quality control.

[0006] To achieve the objectives of this invention, the following technical solution is adopted:

[0007] The first aspect of this invention proposes a method for quality control of open sets of otoscope images, comprising the following steps:

[0008] Construct a quality control training set, which includes positive samples and negative samples. The positive samples are valid otoscopy images that meet the diagnostic criteria, and the negative samples include at least one of the following: images of adjacent anatomical sites, endoscopic images of other departments, and general visual interference images.

[0009] Using the quality control training set, a lightweight convolutional neural network is used as the backbone network and trained through a binary classification loss function to obtain a binary classification verification network.

[0010] Obtain the otoscope image to be diagnosed, input the otoscope image to be diagnosed into the binary classification verification network, and output a valid probability representing that the image is a valid otoscope image.

[0011] A further improvement is that the method further includes the following steps:

[0012] The valid probability is compared with a preset rejection threshold. If the valid probability is less than the rejection threshold, the image is determined to be invalid and a re-acquisition prompt is issued. If the valid probability is greater than or equal to the rejection threshold, the image is determined to be valid and allowed to proceed to the subsequent diagnostic stage.

[0013] Further improvements include the following: the images of adjacent anatomical sites include nasal endoscopy images and laryngoscopy images that have similar tubular field of view and mucosal texture to otoscopy images, as well as external auditory canal images that do not expose key anatomical landmarks of the tympanic membrane; the other endoscopic images include gastroscopy images or colonoscopy images with different anatomical structures and mucosal features; and the general visual interference images include manipulation artifact images, overexposed / underexposed images, and non-medical scene images.

[0014] A further improvement is that the training phase of the binary classification verification network includes the following steps:

[0015] The backbone network of the lightweight convolutional neural network is initialized using weights pre-trained on a general image dataset, retaining all pre-trained parameters except for the classification head.

[0016] The training samples are input into the network with a set batch size for forward propagation, and then pass through the feature extraction layer and the classification head to obtain the predicted output. The binary classification loss function value between the predicted output and the true label is calculated.

[0017] An adaptive moment estimation optimizer is used to backpropagate based on the loss function value, calculate the gradient of the parameters of each layer of the network, and update the network parameters with the set initial learning rate and weight decay coefficient.

[0018] A learning rate decay strategy is adopted to update the learning rate in each training round or every few rounds, so that the learning rate gradually decreases as the training progresses until the preset training termination condition is reached.

[0019] A further improvement is that the lightweight convolutional neural network is an EfficientNet series network, and its network structure includes:

[0020] The input layer receives the scaled and normalized RGB image.

[0021] The feature extraction layer consists of multiple moving flip bottleneck convolutional modules stacked sequentially. Each module contains depthwise separable convolution, batch normalization, squeeze-excited attention mechanism and point convolution, which are used to extract hierarchical deep features from the input image and finally output a feature map with a preset number of channels and spatial resolution.

[0022] The classification head includes a global pooling layer and a fully connected layer. The global pooling layer is used to compress the spatial dimension of the feature map into a one-dimensional feature vector. The fully connected layer is used to map the one-dimensional feature vector into a scalar logit value, which represents the original confidence that the input image belongs to a valid otoscope image.

[0023] A further improvement is that the effective probability is obtained by applying a Sigmoid nonlinear mapping to the logit value output by the binary classification verification network, wherein the Sigmoid mapping converts any real logit value into a probability value between 0 and 1.

[0024] A further improvement is that the binary classification verification network also outputs a confidence assessment associated with the effective probability during the inference phase, specifically including:

[0025] Calculate the absolute difference between the valid probability and the rejection threshold. When the absolute difference is less than a preset fuzzy interval threshold, determine that the current sample belongs to the boundary fuzzy sample.

[0026] In response to the determination, the corresponding image is automatically marked as "requires manual review" and temporarily stored in the review queue. At the same time, a prompt message is issued to trigger the manual review process.

[0027] A further improvement is that, before inputting the otoscope image to be diagnosed into the binary classification verification network, a standardization preprocessing step is included: scaling the short side of the original image to the target size using an interpolation algorithm while maintaining the original aspect ratio; cropping a region of interest of a preset size based on the image center; and normalizing the cropped image to make its pixel value distribution match the input requirements of the binary classification verification network.

[0028] A second aspect of the present invention provides an open set quality control system for otoscope images, comprising:

[0029] A quality control training set construction module is used to construct a quality control training set, which includes positive samples and negative samples. The positive samples are valid otoscope images that meet the diagnostic criteria, and the negative samples include at least one of the following: images of adjacent anatomical sites, endoscopic images of other departments, and general visual interference images.

[0030] The network training module is used to train a binary classification verification network by using the quality control training set, employing a lightweight convolutional neural network as the backbone network, and using a binary classification loss function.

[0031] The image acquisition module is used to acquire an otoscope image to be diagnosed and input the otoscope image to be diagnosed into the binary classification verification network;

[0032] The reasoning and judgment module is used to output a valid probability that an image is a valid otoscope image through the binary classification verification network, and compare the valid probability with a preset rejection threshold. If the valid probability is less than the rejection threshold, it is judged as an invalid image and a re-acquisition prompt is issued; if the valid probability is greater than or equal to the rejection threshold, it is judged as a valid image and allowed to proceed to the subsequent diagnostic stage.

[0033] A third aspect of the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a method for quality control of an open set of otoscope images as described in any one of the first aspects.

[0034] The beneficial effects of this invention are as follows:

[0035] This invention addresses the technical challenges of inadequate clinical quality control of otoscopy images and the susceptibility of models to "hallucination" predictions. By constructing a quality control training set containing positive samples and multiple classes of negative samples, the model can actively identify invalid inputs such as adjacent anatomical images, other endoscopic images, and common visual interference, fundamentally overcoming the limitations of traditional closed-set models. By employing a lightweight convolutional neural network as the backbone, it achieves a high interception rate while maintaining an extremely low false positive rate, effectively protecting valid images from being mistakenly blocked.

[0036] This invention, with extremely low computing power and storage overhead, blocks erroneous diagnoses caused by invalid images at the source, significantly improving clinical safety and reliability, and providing key technical support for the reliable implementation of intelligent otoscopy diagnosis in primary healthcare and large-scale screening scenarios. Attached Figure Description

[0037] Figure 1 This is an overall flowchart of the open set quality control method for otoscope images according to the present invention;

[0038] Figure 2 This is a schematic diagram of an electronic device according to the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0040] Please refer to the attached document. Figure 1 -Appendix Figure 2 This invention proposes a method for quality control of open sets of otoscope images, such as... Figure 1 As shown, it includes the following steps:

[0041] Step S1: Construct a quality control training set, which includes positive samples and negative samples. The positive samples are valid otoscopy images that meet the diagnostic criteria, and the negative samples include at least one of the following: images of adjacent anatomical sites, endoscopic images from other departments, and general visual interference images.

[0042] Step S2: Using the quality control training set, a lightweight convolutional neural network is used as the backbone network and trained through a binary classification loss function to obtain a binary classification verification network.

[0043] Step S3: Obtain the otoscope image to be diagnosed, input the otoscope image to be diagnosed into the binary classification verification network, and output a valid probability representing that the image is a valid otoscope image.

[0044] Step S4: Compare the valid probability with the preset rejection threshold. If the valid probability is less than the rejection threshold, the image is determined to be invalid and a re-acquisition prompt is issued. If the valid probability is greater than or equal to the rejection threshold, the image is determined to be valid and allowed to proceed to the subsequent diagnostic stage.

[0045] The method of the present invention will now be described in more detail.

[0046] Specifically, in step S1, the quality control training set constructed by the present invention includes positive samples and negative samples. Positive samples were defined as clinically confirmed valid endoscopic images of the ear, sourced from the ENT outpatient departments of multiple centers (Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Sun Yat-sen Memorial Hospital Shenzhen-Shantou Central Hospital of Sun Yat-sen University, the Fifth Affiliated Hospital of Southern Medical University, and Zhongshan People's Hospital), totaling 17,352 images. These images covered 22 pre-defined ear diseases, including: normal ear, cerumen in the external auditory canal, external auditory canal mass, external auditory canal osteoma, external auditory canal cholesteatoma, external auditory canal foreign body, external auditory canal hair, fungal external otitis, granulomatous myringitis, bullous myringitis, tympanic membrane calcification, epitympanic retraction bag, incomplete tympanic membrane expansion, secretory otitis media, post-tympanostomy tube insertion, middle ear cholesterol granuloma, acute otitis media, chronic suppurative otitis media, adhesive otitis media, epitympanic cholesteatoma, adhesive cholesteatoma, and middle ear cholesteatoma.

[0047] Image acquisition equipment included a 0° rigid otoscope (Karl Storz, 2.7mm in diameter) and a portable video otoscope (Shenzhen Black Bee Technology, 3.1mm in diameter), with resolutions ranging from 500×500 to 700×700 pixels. All positive samples were independently annotated and confirmed by three senior physicians with over 10 years of clinical experience in otology, and controversial images were excluded.

[0048] A total of 14,641 negative samples were constructed according to the following three categories to simulate various invalid inputs that may occur in real clinical scenarios:

[0049] Category 1: Images of adjacent anatomical sites. This includes nasal endoscopy images (nasal cavity, turbinates, nasopharynx, with tubular views and mucosal textures similar to otoscopy images), laryngoscopy images (vocal cords, epiglottis), and images of the external auditory canal excluding the tympanic membrane (images showing only the skin, cerumen, or hair follicles of the external auditory canal, without revealing key features of the tympanic membrane). These samples have extremely high visual similarity to positive samples and are the most difficult interference items for the model to distinguish.

[0050] Category 2: Endoscopic images from other departments. This includes gastroscopy images (e.g., esophageal, gastric antrum, intestinal mucosa) and colonoscopy images, which have anatomical structures and mucosal features completely different from otoscopy images.

[0051] Category 3: General visual interference images. These include operation artifacts (such as fingers blocking the lens, lens smudges, completely overexposed or completely underexposed black and white images), and non-medical scenes (such as background photos of a doctor's work environment, overall photos of medical equipment, and images of completely unrelated natural landscapes or objects).

[0052] All negative samples were manually reviewed to ensure they did not contain any valid endoscopic anatomical structures of the ear. The sampling ratio of positive to negative samples was set to 1:1, with the three types of negative samples sampled approximately proportionally in the training set. The training and validation sets were divided in an 8:2 ratio: the training set contained 25,549 images, and the validation set contained 6,444 images.

[0053] Specifically, in step S2, this invention uses EfficientNet-B0 as the lightweight convolutional neural network backbone to construct a binary classification verification network. The network structure includes:

[0054] Input layer: Receives RGB images, uniformly scales them, and centers them to 224×224 pixels. For images with varying original resolutions, bilinear interpolation is used for scaling.

[0055] Feature extraction layer: EfficientNet-B0 balances network depth, width, and resolution using a composite scaling method, with approximately 5.3M parameters. It consists of multiple moving-flipping bottleneck convolutional modules (MBConv) stacked sequentially. Each module executes the following in sequence: depthwise separable convolution, batch normalization, squeeze-encouragement attention (SE Attention), pointwise convolution, and residual connections (when the number of input and output channels is the same). The final output is a feature map with a dimension of 7×7×1280.

[0056] Classification Head: Global Average Pooling is performed on the feature map to compress the 7×7×1280 spatial features into a 1×1280 one-dimensional feature vector; then a single-node fully connected layer (Linear (mapping 1280 dimensions to 1 dimension)) is connected to output a scalar logit value, which represents the original confidence that the input image belongs to the positive class (valid otoscope image).

[0057] The specific process for the training phase is as follows:

[0058] Parameter initialization: Load the EfficientNet-B0 weight file pre-trained on the ImageNet-1K dataset, and copy the parameters of all convolutional layers, batch normalization layers, and fully connected layers in it to the corresponding layers of this network. Since the output dimension of the classification head of this network is 1 (the original pre-trained model's classification head output dimension was 1000), the parameters of the fully connected layers of the classification head are randomly initialized (normal distribution with a mean of 0 and a standard deviation of 0.01).

[0059] Forward propagation and loss calculation: Training samples are input into the network at a set batch size (e.g., 32), passing sequentially through the feature extraction layer and the classification head to obtain the predicted logit value z. A sigmoid function is applied to z to obtain the predicted probability. The loss value is calculated using the binary cross-entropy loss function, and the formula is as follows:

[0060]

[0061] in For batch size, The true labels are (1 represents a positive sample, and 0 represents a negative sample). To predict probabilities.

[0062] Backpropagation and parameter update: Using the AdamW optimizer, set the initial learning rate (e.g., ), weight decay coefficient (e.g.) Backpropagation is performed based on the loss function value to calculate the gradient of the parameters of each layer, and the network parameters are updated using the AdamW update formula.

[0063] Learning rate scheduling: A cosine annealing learning rate scheduling strategy is adopted, with a total of 50 training epochs. The learning rate varies with the number of training epochs t as follows:

[0064]

[0065] in The initial learning rate, This is the final learning rate (which can be set to 0). This represents the total number of training epochs. The learning rate is updated after each epoch, and the model with the minimum loss on the validation set is saved as the final model.

[0066] Regularization: During training, a dropout rate (e.g., 0.1) is set for the Dropout layer within each MBConv module; simultaneously, a weight decay (L2 regularization) coefficient (e.g., 1×10⁻⁶) is applied. -4 (to prevent overfitting).

[0067] Specifically, in steps S3 and S4, during the reasoning phase, for the otoscope image to be diagnosed, the following steps are performed:

[0068] Standardization preprocessing: The original image is read, its height and width are obtained, the length of the shorter side is calculated, and the shorter side is scaled to 256 pixels using bilinear interpolation while maintaining the original aspect ratio. Using the center point of the scaled image as a reference, a 224×224 pixel square region of interest is cropped outwards, ensuring that the cropped region completely covers the tympanic membrane and surrounding key anatomical structures. Pixel value normalization is performed on the cropped image: pixel values ​​are linearly mapped from [0,255] to the [0,1] interval, then the preset mean vector [0.485, 0.456, 0.406] is subtracted, and the result is divided by the preset standard deviation vector [0.229, 0.224, 0.225], making the distribution of the input data fit the training statistical characteristics of EfficientNet-B0.

[0069] Forward inference: The standardized image tensor is input into the trained EfficientNet-B0 binary classification and verification network, passing sequentially through the feature extraction layer and the classification head, outputting the logit value z, and the effective probability is calculated using the Sigmoid function. .

[0070] Judgment and Output: Set a rejection threshold T (T=0.5 in this embodiment). If p ≥ T, the image is judged as valid, a "valid" label is output, and it is allowed to proceed to the subsequent diagnostic module; if p < T, the image is judged as invalid, an "invalid" label is output, the quality control process is terminated, and the message "Image invalid, please retake or upload a clear otoscope photo" is returned through the interactive interface.

[0071] To verify the stability of the method under different noise distributions, five test scenarios were constructed: Clean (100% positive samples, 0% negative samples), Low-Noise (90% positive samples, 10% negative samples), Medium-Noise (70% positive samples, 30% negative samples), High-Noise (50% positive samples, 50% negative samples), and Extreme-Noise (20% positive samples, 80% negative samples). Each test group had a fixed number of 3495 positive samples, and the number of negative samples was configured according to the scenario proportions, with each of the three types of interference accounting for one-third of the negative samples. Evaluation metrics included accuracy, precision, recall, F1 score, false positive rate (the proportion of positive samples misclassified as negative), and false negative rate (the proportion of negative samples missed as positive). The experimental results are shown in Table 1.

[0072] Clean 3495 0 3493 0 0 2 3495 99.94% 100.00% 99.94% 99.97% 0.06% 0.00% Low 3495 388 3493 0 388 2 3883 99.95% 100.00% 99.94% 99.97% 0.06% 0.00% Medium 3495 1498 3493 0 1498 2 4993 99.96% 100.00% 99.94% 99.97% 0.06% 0.00% High 3495 3495 3493 0 3495 2 6990 99.97% 100.00% 99.94% 99.97% 0.06% 0.00% Extreme 3495 13980 3493 2 13978 2 17475 99.98% 99.94% 99.94% 99.94% 0.06% 0.01%

[0073] Table 1. Evaluation results of EfficientNet-B0 in five scenarios.

[0074] Table 1 shows the evaluation results of EfficientNet-B0 in five scenarios, where TP represents true positives, FP represents false positives, FN represents false negatives, and TN represents true negatives. As can be seen from the results in Table 1, the model maintains high stability under different noise intensities: the false positive rate for positive samples remains constant at approximately 0.06%, indicating stable protection against positive samples; the missed detection rate is 0% except in extreme scenarios, and even in extreme scenarios it is only 0.01%; the F1 score is close to 1 in all scenarios, indicating that the model has strong robustness to changes in distribution.

[0075] To verify the lightweight advantage of EfficientNet-B0 in this invention, it was compared with ConvNeXt-Tiny (approximately 28M) which has a larger number of parameters. The training configurations were completely identical, and the results are shown in Table 2.

[0076] Clean 3495 0 3492 0 0 3 3495 99.91% 100.00% 99.91% 99.96% 0.09% 0.00% Low 3495 388 3492 1 387 3 3883 99.90% 99.97% 99.91% 99.94% 0.09% 0.26% Medium 3495 1498 3492 1 1497 3 4993 99.92% 99.97% 99.91% 99.94% 0.09% 0.07% High 3495 3495 3492 1 3494 3 6990 99.94% 99.97% 99.91% 99.94% 0.09% 0.03% Extreme 3495 13980 3492 21 13959 3 17475 99.86% 99.40% 99.91% 99.66% 0.09% 0.15%

[0077] Table 2 Evaluation results of ConvNeXt-Tiny in five scenarios

[0078] Compared to ConvNeXt-Tiny, the EfficientNet-B0 adopted in this invention has the following advantages: the false positive rate is reduced from 0.09% to 0.06%, a reduction of 33%; in Low / Medium / High scenarios, the missed block rate is reduced from non-zero (maximum 0.26%) to 0%; in Extreme scenarios, the missed block rate is reduced from 0.15% to 0.01%, a reduction of 93%; the model size is compressed from approximately 319MB (checkpoint files saved by ConvNeXt-Tiny) to 47MB, a compression ratio of approximately 85%, significantly reducing deployment costs. This demonstrates that the lightweight design of this invention, while ensuring or even improving quality control performance, significantly reduces storage and computational overhead, enabling efficient operation on low-computing-power edge devices (such as NVIDIA Jetson Nano and mobile devices).

[0079] Understandably, the core of this invention is to build a lightweight binary classification network based on EfficientNet-B0, which breaks through the limitation of closed sets and actively intercepts non-otoscope and low-quality images through a first-verification and then-diagnosis mechanism. This blocks algorithmic illusions and serious misdiagnosis from the source, achieving efficient filtering with extremely low computational overhead. It not only optimizes the system response speed, but also standardizes the image acquisition process through real-time feedback.

[0080] In a preferred embodiment of this invention, during the inference phase, in addition to outputting the effective probability p, the invention also outputs a confidence assessment associated with the effective probability. Specifically:

[0081] Calculate the absolute difference d = |p - T| between the valid probability p and the rejection threshold T (T=0.5). Assume the fuzzy interval threshold δ=0.1. When d <δ (i.e., p is in the interval [0.4, 0.6)), the system determines that the current sample belongs to the boundary fuzzy sample, and the model's confidence level is insufficient.

[0082] In response to this determination, the system automatically marks the corresponding image as requiring manual review and temporarily stores it in the review queue. Simultaneously, it issues a prompt through the interactive interface: "The AI's confidence level for this image quality boundary is low; manual review is recommended." Physicians can access a separate window to view the image and make a final judgment. Understandably, this mechanism effectively reduces the risk of automatic misjudgment of boundary samples, making it particularly suitable for quality control of blurry or atypical images in clinical practice.

[0083] In this embodiment, the rejection threshold is a preset fixed value T=0.5, which is determined by grid search on the validation set.

[0084] The open-set quality control method for otoscope images provided by this invention achieves efficient, robust, and lightweight quality control of otoscope images by constructing a multi-level open-set negative sample library, employing a lightweight EfficientNet-B0 network, and introducing a boundary sample manual review mechanism. Experimental data shows that in a simulated clinical open-set scenario (negative sample rate 13.5%), the interception rate reaches 98.7%, and the false positive rate is only 0.09%. Compared with ConvNeXt-Tiny, the false positive rate is reduced from 0.09% to 0.06%, the missed interception rate is reduced from 0.15% to 0.01%, and the model size is compressed from 319MB to 47MB. The boundary sample manual review mechanism further reduces the risk of misjudgment of blurred images.

[0085] Compared with existing technologies, this invention provides a specialized input security protection method for otoscopy AI diagnostic systems for the first time, effectively blocking model hallucination predictions and having significant clinical application value.

[0086] The second aspect of this invention provides an open set quality control system for otoscope images, which corresponds to the open set quality control method for otoscope images provided in the above-mentioned embodiments of this invention. Since the open set quality control system for otoscope images provided in this invention corresponds to the open set quality control method for otoscope images provided in the above-mentioned embodiments of this invention, the implementation method of the aforementioned open set quality control method for otoscope images is also applicable to the open set quality control system for otoscope images provided in this embodiment, and will not be described in detail here.

[0087] The system mainly includes the following modules:

[0088] A quality control training set construction module is used to construct a quality control training set, which includes positive samples and negative samples. The positive samples are valid otoscopy images that meet the diagnostic criteria, and the negative samples include at least one category selected from images of adjacent anatomical sites, endoscopic images of other departments, and general visual interference images.

[0089] The network training module is used to train a binary classification verification network by using the quality control training set, employing a lightweight convolutional neural network as the backbone network, and using a binary classification loss function.

[0090] The image acquisition module is used to acquire an otoscope image to be diagnosed and input the otoscope image to be diagnosed into the binary classification verification network.

[0091] The reasoning and judgment module is used to output a valid probability that an image is a valid otoscope image through the binary classification verification network, and compare the valid probability with a preset rejection threshold. If the valid probability is less than the rejection threshold, it is judged as an invalid image and a re-acquisition prompt is issued; if the valid probability is greater than or equal to the rejection threshold, it is judged as a valid image and allowed to proceed to the subsequent diagnostic stage.

[0092] See Figure 2 The present invention also provides an electronic device and a computer-readable storage medium.

[0093] like Figure 2 The diagram illustrates an electronic device according to an embodiment of the present invention. This electronic device includes a processor 11, a memory 12, and a computer program stored in the memory and executable on the processor 11. When the processor 11 executes the computer program, it implements the steps described in the embodiment of the open set quality control method for otoscope images. Alternatively, when the processor 11 executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0094] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor 11 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0095] The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the schematic diagram is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0096] The processor 11 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0097] The memory 12 can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system 121, at least one application program 122 required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0098] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0099] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A method for quality control of open sets of otoscope images, characterized in that, Includes the following steps: Construct a quality control training set, which includes positive samples and negative samples. The positive samples are valid otoscopy images that meet the diagnostic criteria, and the negative samples include at least one of the following: images of adjacent anatomical sites, endoscopic images of other departments, and general visual interference images. Using the quality control training set, a lightweight convolutional neural network is used as the backbone network and trained through a binary classification loss function to obtain a binary classification verification network. Obtain the otoscope image to be diagnosed, input the otoscope image to be diagnosed into the binary classification verification network, and output a valid probability representing that the image is a valid otoscope image.

2. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, The method further includes the following steps: The valid probability is compared with a preset rejection threshold. If the valid probability is less than the rejection threshold, the image is determined to be invalid and a re-acquisition prompt is issued. If the valid probability is greater than or equal to the rejection threshold, the image is determined to be valid and allowed to proceed to the subsequent diagnostic stage.

3. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, The images of adjacent anatomical sites include nasal endoscopy images and laryngoscopy images that have similar tubular field of view and mucosal texture to otoscopy images, as well as external auditory canal images that do not expose key anatomical landmarks of the tympanic membrane; the images of other endoscopic departments include gastroscopy images or colonoscopy images with different anatomical structures and mucosal features; the images of general visual interference include operation artifact images, overexposed / underexposed images, and images of non-medical scenes.

4. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, The training phase of the binary classification verification network includes the following steps: The backbone network of the lightweight convolutional neural network is initialized using weights pre-trained on a general image dataset, retaining all pre-trained parameters except for the classification head. The training samples are input into the network with a set batch size for forward propagation, and then pass through the feature extraction layer and the classification head to obtain the predicted output. The binary classification loss function value between the predicted output and the true label is calculated. An adaptive moment estimation optimizer is used to backpropagate based on the loss function value, calculate the gradient of the parameters of each layer of the network, and update the network parameters with the set initial learning rate and weight decay coefficient. A learning rate decay strategy is adopted to update the learning rate in each training round or every few rounds, so that the learning rate gradually decreases as the training progresses until the preset training termination condition is reached.

5. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, The lightweight convolutional neural network is a member of the EfficientNet series, and its network structure includes: The input layer receives the scaled and normalized RGB image. The feature extraction layer consists of multiple moving flip bottleneck convolutional modules stacked sequentially. Each module contains depthwise separable convolution, batch normalization, squeeze-excited attention mechanism and point convolution, which are used to extract hierarchical deep features from the input image and finally output a feature map with a preset number of channels and spatial resolution. The classification head includes a global pooling layer and a fully connected layer. The global pooling layer is used to compress the spatial dimension of the feature map into a one-dimensional feature vector. The fully connected layer is used to map the one-dimensional feature vector into a scalar logit value, which represents the original confidence that the input image belongs to a valid otoscope image.

6. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, The effective probability is obtained by applying a Sigmoid nonlinear mapping to the logit value output by the binary classification verification network. The Sigmoid mapping converts any real logit value into a probability value between 0 and 1.

7. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, The binary classification verification network also outputs a confidence assessment associated with the effective probability during the inference phase, specifically including: Calculate the absolute difference between the valid probability and the rejection threshold. When the absolute difference is less than a preset fuzzy interval threshold, determine that the current sample belongs to the boundary fuzzy sample. In response to the determination, the corresponding image is automatically marked as "requires manual review" and temporarily stored in the review queue. At the same time, a prompt message is issued to trigger the manual review process.

8. The method for quality control of open sets of otoscope images according to claim 1, characterized in that, Before inputting the otoscope image to be diagnosed into the binary classification verification network, a standardization preprocessing step is also included: scaling the short side of the original image to the target size using an interpolation algorithm while maintaining the original aspect ratio; cropping a region of interest of a preset size based on the image center; and normalizing the cropped image to make its pixel value distribution match the input requirements of the binary classification verification network.

9. An open set quality control system for otoscope images, characterized in that, include: A quality control training set construction module is used to construct a quality control training set, which includes positive samples and negative samples. The positive samples are valid otoscope images that meet the diagnostic criteria, and the negative samples include at least one of the following: images of adjacent anatomical sites, endoscopic images of other departments, and general visual interference images. The network training module is used to train a binary classification verification network by using the quality control training set, employing a lightweight convolutional neural network as the backbone network, and using a binary classification loss function. The image acquisition module is used to acquire an otoscope image to be diagnosed and input the otoscope image to be diagnosed into the binary classification verification network; The reasoning and judgment module is used to output a valid probability that an image is a valid otoscope image through the binary classification verification network, and compare the valid probability with a preset rejection threshold. If the valid probability is less than the rejection threshold, it is judged as an invalid image and a re-acquisition prompt is issued; if the valid probability is greater than or equal to the rejection threshold, it is judged as a valid image and allowed to proceed to the subsequent diagnostic stage.

10. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a method for quality control of an open set of otoscope images as described in any one of claims 1 to 8.

Images