Contact lens care product disinfection efficacy identification method based on portable equipment and related device

Abstract

The invention relates to the technical field of eye science, and discloses a contact lens care product disinfection efficacy identification method based on portable equipment and a related device. The method comprises the following steps: updating a first machine model on the portable equipment according to the model and the service life of the portable equipment; carrying out image acquisition and preprocessing so as to output images which are standardized in color and brightness, reduced in noise and geometrically aligned; automatically positioning a thallus counting plate area based on image recognition so as to automatically focus on a counting grid area; acquiring a microscopic image and performing thallus particle identification by using a first machine model; and classifying and identifying viable bacteria and dead bacteria of the identified thallus particles to calculate the disinfection efficacy. According to the invention, an image recognition method which can fuse image acquisition preprocessing and a model adaptation mechanism and support stable operation under the condition of portable equipment is realized, so that objective, rapid and automatic evaluation of the disinfection efficacy of the contact lens care product is realized.

Description

Technical Field

[0001] This application relates to the technical field of ophthalmology, and particularly to a method for identifying the disinfection efficacy of contact lens care products based on portable devices and related devices. Background Art

[0002] As a commonly used vision correction tool, contact lenses rely heavily on care products during use. In particular, the disinfection performance of the care solution is directly related to the eye health of the wearer. Currently, the mainstream methods for evaluating the disinfection efficacy of contact lens care solutions mainly include two categories: one is the traditional culture counting method, that is, culturing the bacterial suspension after disinfection treatment, and then judging the sterilization rate by counting the number of colonies; the other is the manual counting method under a microscope, that is, observing the stained bacterial bodies through a microscope, manually identifying live bacteria and dead bacteria, and counting the quantity. The former takes a long time, has cumbersome operations, and has high requirements for experimental conditions; although the latter shortens the cycle, it still highly relies on manual experience, has poor repeatability, is easily interfered by subjective factors, and is not conducive to real-time monitoring in large-scale and standardized production environments.

[0003] To overcome the deficiencies of traditional methods, existing technologies have proposed a solution that combines thiazolyl blue staining and a bacterial counting plate to achieve the distinction between live / dead bacteria and simple image recognition through digital photography. Some methods attempt to collect images using a digital camera or scanning device and then analyze them through subsequent software tools. However, these methods still mainly use laboratory or desktop devices as the main usage scenarios, lack practicality for ordinary consumers or mobile terminal scenarios, and usually do not have the ability to adapt to changes in image acquisition conditions, such as image degradation caused by different types of shooting devices, differences in lighting conditions, and lens aging.

[0004] In addition, although deep learning technology has been widely applied in the field of image recognition, effectively applying this technology to microscopic image processing in a consumer-grade portable device environment still faces many challenges. On the one hand, the imaging quality of portable devices (such as smartphones) is unstable, and the imaging conditions are limited, resulting in significant changes in the input images; on the other hand, the optical characteristics vary significantly among different models of devices, and the imaging ability of the same device shows a trend of attenuation over time, making it difficult for traditional static models to adapt in the long term. Summary of the Invention

[0005] In order to implement an image recognition method that can integrate image acquisition preprocessing and model adaptation mechanisms and support stable operation under portable device conditions, so as to achieve an objective, fast, and automated evaluation of the disinfection efficacy of contact lens care products, this application provides a method for identifying the disinfection efficacy of contact lens care products based on portable devices and related devices.

[0006] In a first aspect, the present application provides a method for identifying the disinfection efficacy image of a contact lens care product based on a portable device, adopting the following technical solutions:

[0007] A method for identifying the disinfection efficacy image of a contact lens care product based on a portable device, comprising the following steps:

[0008] S1. Update the first machine model on the portable device according to the model and service life of the portable device; wherein, the first machine model is used to identify bacterial particles, and the portable device includes a smartphone camera and a macro / micro lens attachment;

[0009] S2. Perform image acquisition and preprocessing to achieve an image that can output color and brightness normalization, noise reduction, and geometric alignment;

[0010] S3. Automatically locate the bacterial counting plate area based on image recognition to automatically focus on the counting grid area;

[0011] S4. Obtain a microscopic image and use the first machine model to identify bacterial particles; wherein, the microscopic image is obtained by microscopically photographing the portable device after the nursing solution and the bacterial suspension are mixed, stained with a stain for bacteria or fungi, and dropped into the micro-grid area of the bacterial counting plate;

[0012] S5. Classify and identify the identified bacterial particles into live bacteria and dead bacteria to calculate the disinfection efficacy.

[0013] Optionally, S1 includes the following steps:

[0014] S11. Read the model and service life of the portable device;

[0015] S12. Use the portable device to perform image acquisition and preprocessing on a number of standard slides; wherein, the standard slides correspond to pre-entered standard images in the database, and the coordinate positions, live / dead bacteria labels, color feature labels, and clarity labels of the bacteria in the standard images are pre-annotated;

[0016] S13. Retrain the pre-trained first machine model stored on the portable device based on the images acquired by the portable device, the corresponding pre-entered standard images in the database, and the relevant annotations;

[0017] S14. After the retraining of the machine model is completed, use a preset standard slide as the validation set to evaluate the performance difference between the retrained machine model and the machine model before retraining; if the performance of the retrained machine model is better than that of the machine model before retraining, then apply the retrained machine model to the portable device and deploy it; otherwise, maintain the use of the machine model before retraining.

[0018] Optionally, S13 includes the following steps:

[0019] S131. Retrieve the pre-trained first machine model stored on the portable device, and freeze the front-end convolutional layer and the intermediate layer;

[0020] S132. Add a number of adaptation layers after the frozen front-end convolutional layer and intermediate layer as device adaptation layers; among them, the number and scale of the device adaptation layers are determined according to the mobile phone model and the service life;

[0021] S133. Individually train the newly added adaptation layers using the standard slide image uploaded by the user, where a relatively high learning rate is used for the individual training;

[0022] S134. Unfreeze the basic network parameters, and then perform a short-cycle overall fine-tuning using the same standard slide image, where a relatively low learning rate is used for the fine-tuning.

[0023] Optionally, the said S2 includes the following steps:

[0024] S21. Output user operation prompts to guide the user to align with the grid area of the bacterial counting plate;

[0025] S22. Perform color correction and white balance on the image input;

[0026] S23. Perform noise reduction and enhancement on the image input;

[0027] S24. Perform geometric correction on the image input.

[0028] Optionally, S3 includes the following steps:

[0029] S31. Input the pre-processed image into the trained lightweight segmentation network, and the network output is the segmentation region probability map;

[0030] S32. Perform mask outer boundary rectangle extraction and image cropping.

[0031] Optionally, the said S4 includes the following steps:

[0032] Input the pre-processed microscopic image into the first machine model, and the network output is the recognition result of the positions of bacterial particles.

[0033] Optionally, the said S5 includes the following steps:

[0034] S51. Identify live bacteria and dead bacteria based on the stained color and morphological characteristics;

[0035] S52. Count the number of live bacteria and dead bacteria, and calculate the proportion to calculate the disinfection efficacy.

[0036] Optionally, the pre-training steps of the said first machine model include:

[0037] S101. Obtain microscopic images of standardized glass slides taken by a portable device of a specific model, where the microscopic images are multiple and cover different lighting conditions, different shooting angles, different sample concentrations, and staining concentrations;

[0038] S102. Perform annotation and data augmentation on the microscopic images. Among them, the annotation includes: the position and survival status of each bacterial / fungal particle, and the precise pixel mask of the grid area;

[0039] S103. Perform initialization of the network structure and parameter settings, and divide the dataset of microscopic images into a training set, a validation set, and a test set;

[0040] S104. Use the training set and the validation set to perform full-data training on the initialized model to obtain a first machine model for identifying bacterial particles.

[0041] In a second aspect, the present application provides a method for identifying the disinfection efficacy image of a contact lens care product based on a portable device, adopting the following technical solution:

[0042] A method for identifying the disinfection efficacy image of a contact lens care product based on a portable device includes a processor, and a program of the method for identifying the disinfection efficacy image of a contact lens care product based on a portable device described in any one of the above is run in the processor.

[0043] In a third aspect, the present application provides a storage medium, adopting the following technical solution:

[0044] A storage medium stores a program of the method for identifying the disinfection efficacy image of a contact lens care product based on a portable device described in any one of the above.

[0045] In summary, the present application includes at least one of the following beneficial technical effects:

[0046] 1. The present application realizes the active identification of the model and usage years of portable devices, and constructs an adaptability retraining mechanism based on standard slide images, enabling the bacterial identification model to dynamically adjust according to the imaging characteristics of different devices, thereby improving the stability and identification accuracy of the model during long-term use.

[0047] 2. Adopt a recognition architecture with clear division of labor, separate the modeling of bacterial particle recognition and live / dead classification, make the model structure concise and the training target clear, and facilitate deployment in a resource-constrained mobile environment to meet the actual requirements of speed and energy consumption in consumer-level application scenarios.

[0048] 3. The system automatically determines the live / dead state of bacteria through staining feature modeling, and combines quantity statistics to output the disinfection efficacy index, realizing the full-process automatic analysis from microscopic image input to the quantification of the bactericidal effect of the nursing solution. It has the advantages of high efficiency, accuracy, and strong repeatability, and is significantly superior to the existing manual recognition and traditional culture methods. Description of the Drawings

[0049] Figure 1 It is a flowchart of a method for image recognition of the disinfection efficacy of a contact lens care product based on a portable device in an embodiment of the present application. Detailed Embodiments

[0050] The following details the embodiments of the present application, and the examples of the embodiments are shown in the drawings.

[0051] In the description of this specification, the descriptions referring to the terms "certain embodiments", "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0052] The embodiments of the present application disclose a method for image recognition of the disinfection efficacy of a contact lens care product based on a portable device. Referring to Figure 1 , the following steps S1 - S5.

[0053] S1. Update the first machine model on the portable device according to the model and service life of the portable device; wherein, the first machine model is used to identify bacterial particles, and the portable device includes a smartphone camera and a macro / micro lens attachment.

[0054] Contact lens care products are special liquid preparations used for cleaning, disinfecting, and storing contact lenses. Their main function is to remove protein deposits on the lens surface and kill attached bacteria or fungi, ensuring that the lens is in a clean and sterile state before wearing. If the disinfection efficacy of the care product is insufficient, it may lead to the residue of pathogens, which can further cause eye infections, keratitis, and even more serious vision damage. Therefore, accurately and conveniently evaluating the disinfection effect of the care solution is of great significance for ensuring the safety of consumers. This efficacy is mainly distinguished by the staining of bacteria in microscopic images - after the treated sample is treated with a staining agent, live bacteria and dead bacteria show different color or morphological reactions in the image. For example, under the condition of thiazolyl blue staining, live bacteria retain the reducing ability and will be stained dark, while dead bacteria are light in color or have no obvious staining, so the survival state of bacteria can be judged by the color difference in the image.

[0055] In this solution, the acquisition of images relies on portable devices, usually smart terminals equipped with a camera module, such as smartphones, tablet devices, etc. When used in conjunction with a detachable macro or simple microscopic lens attachment, high-magnification imaging of the stained bacteria on the bacteria counting plate can be achieved. Due to the general availability, ease of operation, and strong computing power of these devices, they are very suitable for deploying disinfection efficacy recognition methods in consumer-grade scenarios. However, at the same time, the imaging ability of portable devices is not constant. The device model and service life are two key variables that have an important impact on the quality and consistency of microscopic images.

[0056] First of all, there are significant differences in the composition of the imaging module, image processing algorithms, image signal processors (ISPs), etc. among different models of portable devices. For example, the sensor sizes, pixel arrangements, color restoration methods, and automatic white balance strategies adopted by different mobile phone models are not the same, resulting in differences in the brightness, color deviation, clarity, and edge details of the finally generated microscopic images even when shooting under the same sample and equal lighting conditions. For example, mobile phone model A tends to present a cold color tone in microscopic images, resulting in a color shift in the staining of bacteria, while model B may automatically enhance the contrast and strengthen the background interference, thereby affecting the accuracy of image recognition.

[0057] Secondly, during the long-term use of devices of the same model, their imaging quality may also change due to hardware aging. On the one hand, the surface of the lens may be scratched or attached with fine dust due to frequent use, resulting in blurred, glare or spot phenomena in the image; on the other hand, the photosensitive element may have a decreased photosensitive ability over time, causing low overall brightness or increased noise. Although these changes do not affect daily photo-taking, they may lead to blurred image edges and lost details in microscopic image recognition, interfering with the task of bacterial identification. For example, the clarity of the standard slide image taken by the same mobile phone one year after purchase has decreased, resulting in the blurred edges of the particles that could originally be clearly identified, thus reducing the recognition accuracy of the machine model.

[0058] Therefore, in order to ensure that the machine model can stably and accurately identify bacterial particles under different devices and at different time nodes, it is necessary to dynamically adjust the recognition model based on the model and service life of the device. In the present invention, by reading the model and the used years of the portable device as device characteristic parameters, the retraining process of the first machine model is guided, and the adaptation of the model to the specific device state is completed. As a key component for locating and identifying bacterial particles in microscopic images, the accuracy of the first machine model directly determines the reliability of subsequent live / dead bacteria classification and disinfection efficacy calculation. Through this update mechanism, the model can automatically adjust the recognition strategy with the change of the device, improve the robustness of the model to different image quality inputs, and finally achieve a stable and consistent particle recognition effect.

[0059] Specifically, in one embodiment, the pre-training steps of the first machine model include S101-S104.

[0060] S101. Obtain microscopic images of standardized slides taken by a portable device of a specific model, where the microscopic images are multiple and cover different lighting conditions, different shooting angles, different sample concentrations and staining concentrations.

[0061] In order to obtain a model with generalization ability and suitable for deployment on portable devices, pre-training needs to be carried out based on a standardized data set in the initial training stage. This pre-training process first depends on the acquisition of microscopic images of standard slides. The standard slide is a prefabricated microscopic detection sample, on which a bacterial suspension with a known concentration and staining state is dropped. After fixation and staining treatment, it has a stable bacterial distribution, staining condition and color contrast. When collecting images, use a portable device of the target model to take pictures under various lighting conditions (such as direct white light, LED cold light), different angles (front view, slightly tilted 5°-10°) and different sample concentrations and staining depths. The obtained images comprehensively cover the actual imaging environment of the device.

[0062] S102. Annotate and perform data augmentation on the microscopic images, where the annotation includes: the position and survival status of each bacterial / fungal particle, and the precise pixel mask of the grid area.

[0063] After data acquisition, systematic annotation and data augmentation processing need to be performed on the microscopic images. The annotation content includes the position information of each bacterial or fungal particle (such as the rectangular box or the coordinates of the center point) and its survival status label (such as live bacteria or dead bacteria). The survival status here is mainly used for training set screening and verification reference in this step and is not used for the recognition task of the first machine model. In addition, precise pixel-level mask annotation is also performed on the grid area of the bacterial counting plate as auxiliary information for subsequent image preprocessing and region extraction. After completing the annotation, to improve the robustness of the model, a series of augmentation operations also need to be performed on the image data, including rotation, scaling, brightness perturbation, color jitter, blur simulation, etc., so that the model can fully encounter various imaging errors and interference factors during the training phase to enhance its adaptability.

[0064] S103. Perform initialization of the network structure and parameter settings, and divide the dataset of microscopic images into a training set, a validation set, and a test set.

[0065] The preprocessed image data will be divided into a training set, a validation set, and a test set, usually in the ratio of 8:1:1. The training set is used for the fitting optimization of the main model weights, the validation set is used to monitor the overfitting risk and assist in adjusting hyperparameters, and the test set is used for the objective evaluation of the final model performance.

[0066] S104. Use the training set and the validation set to perform full-data training on the initialized model to obtain the first machine model for bacterial particle recognition.

[0067] Full-data training means taking the training set and the validation set as inputs and performing systematic parameter optimization on the initially constructed neural network model, so that the model can accurately learn the feature representations of bacterial particles under different image conditions, and then achieve stable detection output. The training set used covers different lighting conditions, shooting angles, staining intensities, and bacterial densities to ensure that the data distribution used for training is representative; while the validation set is used to evaluate the performance of the model on unseen samples in real time to detect whether there is overfitting.

[0068] During the network training process, Mini-batch Gradient Descent is adopted in combination with an adaptive optimizer such as AdamW, along with a standard loss function (such as focal loss or weighted cross-entropy), to optimize the model's ability to recognize small targets. Focal loss is particularly suitable for small target recognition problems. When dealing with targets such as bacterial cells that are highly dense and easily confused, it can enhance the model's sensitivity to difficult-to-recognize particles. The training process of the model usually includes several epochs. In each epoch, forward and backward propagation are performed on all the images in the training set once to adjust the weight parameters in the model. The Early Stopping mechanism is also applied during training. Once the performance metrics (such as IoU or Recall) on the validation set do not improve over multiple epochs, the training is terminated early to prevent overfitting.

[0069] The network structure uses a lightweight detection architecture adapted to microscopic image tasks, such as YOLOv5-Nano or its pruned version, which has a high frame rate inference speed and a small model size, making it very suitable for deployment on portable devices such as mobile phones. During the training process, the model gradually learns features such as the spatial distribution, edge contours, staining depth, and color texture of bacterial cell particles, and optimizes the convolutional kernel weights of each layer through a layer-by-layer backpropagation mechanism, making the output recognition bounding box gradually approach the annotated bounding box. For example, when faced with dead bacterial cell particles with slight staining and unclear color differences, the model can still accurately distinguish their boundaries from the background by comparing the color channel distribution and local morphological features, and output a stable localization result.

[0070] After a complete training cycle, the finally obtained first machine model can stably output the position recognition results of bacterial cell particles and has the ability to maintain recognition consistency under various shooting conditions of the same model of portable device. The training output of this model is not limited to the weight parameters themselves, but also includes inference-related configuration files such as confidence thresholds, non-maximum suppression strategies (NMS), and image input size configurations, enabling it to have a plug-and-play performance during the deployment phase.

[0071] Specifically, in one embodiment, S1 includes the following steps S11 - S14.

[0072] S11. Read the model and service life of the portable device.

[0073] In this solution, portable devices refer to smartphones or tablet terminals used for microscopic image capture, which are used with macro lenses or microscope accessories to capture images of stained bacteria on bacterial counting plates. Different models of devices have significant differences in hardware composition, image signal processor (ISP) algorithms, lens parameters, etc., and the same model of device will also experience optical degradation as it ages, such as lens aging and increased imaging noise, so it is necessary to first obtain the model and age information of the current device.

[0074] This step can be implemented by calling the system interface provided by the operating system. For example, in the Android platform, the device model can be obtained through Build.MODEL, and the first activation time of the device can be inferred from the timestamp in the system file or the OTA installation time to infer the service life; in the iOS system, the model can be obtained through the device identifier (such as iPhone15,2) combined with the Apple device database, and the service life can be estimated with parameters such as battery cycle records and system installation time. This information is automatically read by the application during the initialization phase and passed into the model adaptation process.

[0075] S12. Use a portable device to collect and preprocess images of several standard slides; wherein the standard slides correspond to pre-recorded standard images in the database, and the coordinate positions of the bacteria, live / dead bacteria labels, color feature labels and clarity labels of the standard images are pre-labeled.

[0076] Standard slides refer to microbial stained sample slides that have been processed through a unified process, have known contents and fixed structures. Their bacterial species, concentration, distribution and staining conditions have been strictly controlled during production. They are highly stable and reusable, and are widely used in the calibration and model training of microscopic identification algorithms.

[0077] The image acquisition process uses the user's current portable device (including supporting microscope accessories) to shoot standard slides and follows unified sampling specifications, including unified shooting distance, light source brightness, focusing standards, etc., to ensure that the imaging characteristics during the sampling process are as consistent as possible. After the acquisition is completed, the system automatically performs a standardized preprocessing process on the image. This process is not adjusted for a certain device or individual conditions, but is executed according to fixed parameters to eliminate non-target factors such as color difference, brightness difference, and small angle errors that may be introduced during the shooting process. This standardized process includes white balance correction, color standardization, image geometry correction (such as perspective distortion correction), image contrast enhancement, size unification, and center alignment. All preprocessing operations are performed based on general rules and do not involve personalized learning to ensure that the structure and visual features of the corresponding image in the database are in the same analysis domain.

[0078] For example, a user uses "Galaxy S22" with a microscope accessory of a certain model to capture an image of a standard slide numbered G03. After standardized preprocessing, the image is consistent with the pre-entered image numbered G03 in the database in terms of color, brightness, and size, enabling subsequent differential analysis and model training to be carried out in a unified feature space. At the same time, the corresponding image in the database already contains complete annotation information such as bacterial cell coordinates, live / dead status, color distribution, and clarity level, serving as the standard label for supervised learning. Due to the high consistency and reusability of the standard slide itself, the same slide can be repeatedly used on multiple devices at different times, serving as an adaptation reference for a long time.

[0079] S13. Retrain the pre-trained first machine model stored on the portable device based on the images collected by the portable device, the corresponding pre-entered standard images in the database, and the relevant annotations.

[0080] Specifically, in one embodiment, S13 includes the following steps S131 - S134.

[0081] S131. Retrieve the pre-trained first machine model stored on the portable device and freeze the front-end convolutional layer and the middle layer.

[0082] First, retrieve the first machine model stored in the portable device, which is a neural network model with general bacterial cell particle recognition ability completed through the standard training process. This model usually includes multiple convolutional layers, pooling layers, and feature fusion layers. Its front-end convolutional layer is mainly used to extract the basic features of the image, such as edges, textures, and color distributions, and the middle layer further forms an abstract high-dimensional representation. In S131, the parameters of these two parts are frozen to maintain the model's stable perception ability of the basic structure of bacterial cell particles and not lose the recognition basis of the general morphology due to the retraining process.

[0083] S132. Add several adaptation layers after the frozen front-end convolutional layer and the middle layer as device adaptation layers; among them, the number and scale of the device adaptation layers are determined according to the mobile phone model and the service life.

[0084] After freezing the backbone network, S132 will add device adaptation layers after the output end of the backbone network. These layers, as "post-conversion modules", are used to model the mapping differences between the imaging characteristics of the current device and the base model. The device adaptation layers can be composed of lightweight convolutional layers, fully connected layers, or attention modules, and their number and width are determined according to the device model and usage years. Specifically, for an iPhone device that has been used for 3 years, the system will insert an adaptation layer composed of 3 consecutive convolutional blocks to compensate for image blurring and color shift problems caused by lens aging and make its output closer to the input feature format expected by the base model. The parameters of the adaptation layer are initially random weights and are then driven by the standard slide images collected by the user for training.

[0085] S133. Individually train the newly added adaptation layers using the standard slide images uploaded by the user, where a relatively high learning rate is used for individual training.

[0086] During the training stage of S133, only the device adaptation layers will be updated, and the parameters of the front-end backbone network will be kept frozen. The training objective is to make the output of the adaptation layer accurately map to the known annotation results of the database images. A high learning rate (such as 1e-3) is used for training to accelerate the adaptation speed and strengthen the personalized adjustment ability of the adaptation layer. For example, in a standard image, the position of a certain bacterial cell particle is clear and the staining situation is clear. During training, the adaptation layer should be able to accurately generate feature maps so that the detection output is consistent with the standard annotation. The number of training rounds in this stage is usually short, and an early stopping mechanism is adopted to avoid overfitting.

[0087] S134. Unfreeze the parameters of the base network and then perform a short-period overall fine-tuning using the same standard slide images, where a relatively low learning rate is used for fine-tuning.

[0088] After the training of the adaptation layer converges, enter S134. Unfreeze some intermediate layers and the posterior structure of the base model and perform overall fine-tuning at a relatively low learning rate. The purpose of this is to open up the channel between the device characteristics and the original model and enable the entire model to form internal parameter consistency in the new input domain. For example, by slightly adjusting the convolutional kernel parameters, the base model can adapt to the color distribution shift or local sharpness decrease in the new input, thereby further improving the stability and accuracy of recognition. The learning rate in this fine-tuning stage is generally set to 1e-5 to 1e-6 to prevent damage to the key recognition paths that have been trained.

[0089] S14. After the retraining of the machine model is completed, use the preset standardized slides as the validation set to evaluate the performance difference between the retrained machine model and the machine model before retraining; if the performance of the retrained machine model is better than that of the machine model before retraining, then apply the retrained machine model to the portable device and deploy it; otherwise, continue to use the machine model before retraining.

[0090] The validation set is sourced from preset standard glass slides. The imaging process is consistent with the data used for retraining, but it is not used for parameter optimization during training. Therefore, it can serve as an independent sample set for evaluating the generalization ability of the new model. These preprocessed validation images are input into the old model and the retrained model, respectively, to obtain corresponding recognition outputs. The system calculates common performance metrics such as accuracy, recall, intersection over union (IoU), or F1-score by comparing the model outputs with the labeled tags of the standard images in the database. For example, if a standard glass slide contains 100 bacterial particles and their positions are clearly marked in the labeled data, and model A identifies 95 of them, with 90 being correctly identified, then the accuracy of this model is 90%, the miss rate is 5%, and the error rate is 5%. The same calculation is performed on the retrained model B. If its accuracy is improved to 94%, the retraining result is considered better than the original model.

[0091] To avoid misjudgment, a weighted scoring mechanism or a performance improvement threshold can also be adopted. For example, it is required that the new model exceeds the old model in multiple metrics, or improves by at least a certain percentage (such as 2%) in the main metric. If the evaluation conditions are met, the retrained model is deployed to the current portable device to replace the original model as the core model for subsequent image recognition; if the conditions are not met, the system retains the original model unchanged to avoid performance degradation. In addition, the evaluation results recorded during the validation process can be used for subsequent model version management and optimization of the upgrade strategy. For example, the scale of the adaptation layer or the training strategy can be adjusted according to the performance of multiple adaptations.

[0092] S2. Perform image acquisition and preprocessing to achieve images that can output color and brightness normalization, noise reduction, and geometric alignment.

[0093] Specifically, in one embodiment, S2 includes the following steps S21 - S24.

[0094] S21. Output user operation prompts to guide the user to align with the grid area of the bacterial counting plate.

[0095] The system will guide the user to complete the shooting posture and focusing operations through the graphical interface to ensure that the grid area of the bacterial counting plate in the captured image is centered in the image and a complete, level, and clear field of view is obtained. This guiding method can be real-time contour prompts, auxiliary frame registration, or semi-transparent overlay of the background image, etc., to visually prompt the user to fine-tune the shooting angle or position. For example, when it is detected that the grid is not centered or tilted, the system will prompt "Please pan the device to the right" or "Please adjust the lens angle" until the grid area in the image is centered and aligned.

[0096] S22. Perform color correction and white balance on the image input.

[0097] After image acquisition is completed, color correction and white balance adjustment are performed in S22. Color correction refers to mapping the hue in the original image back to the standard color space, such as the sRGB color gamut, to address color deviation caused by the light source color temperature or the device ISP algorithm during shooting. White balance adjustment takes the grayish-white area of the counting board background as a reference gray card and automatically adjusts the gain coefficients of the R, G, and B channels to make the overall tone of the image neutral and truly restore the staining situation. For example, an image taken under blue light illumination shows a cooler tone, and the white balance algorithm will enhance the red channel and suppress the blue channel to make the color expression of the stained bacteria closer to the actual visual effect.

[0098] S23. Denoise and enhance the image input.

[0099] Denoising mainly removes the sensor noise and the fine texture interference caused by shooting jitter through image smoothing algorithms such as Gaussian filtering and median filtering; enhancement operations include histogram equalization, contrast stretching, edge sharpening, etc., aiming to improve the local contrast and structural clarity of the image, making the edges of the bacteria sharper and the staining situation easier to distinguish. For example, for an image with insufficient light and blurred edges, the system will first perform brightness stretching and then apply the Unsharp Mask sharpening operation to accurately extract the bacteria particles in subsequent recognition.

[0100] S24. Geometrically correct the image input.

[0101] This step identifies the grid structure of the bacteria counting board in the image and restores the tilted or distorted image to a standard front view based on the perspective transformation model, ensuring that the grid grille spacing is consistent and the ratio is correct. This operation is completed by modeling after extracting line segments through a lightweight image transformation network or Hough transformation. For example, if the user slightly tilts the device, causing the counting board to be trapezoidally deformed, the system will correct it to a rectangle through four-point perspective transformation to ensure the parallelism and equidistance of the grid.

[0102] It should be noted that image acquisition and preprocessing mainly standardize the image quality before inputting into the model, and its purposes include:

[0103] 1. Improve the quality and consistency of the input image

[0104] Image preprocessing (such as color correction, white balance, denoising, and geometric correction) aims to eliminate the differences caused by uncertain factors such as the external environment and shooting equipment, and obtain clean, unified, and standardized images to reduce the volatility of the input data.

[0105] 2. Highlight the target information

[0106] The key features of bacteria or fungi (such as staining differences, edge clarity) are highlighted through preprocessing, reducing background interference, and facilitating the model to efficiently learn and identify features.

[0107] 3. Reduce the model burden

[0108] Through standardized preprocessing, subsequent model tasks can be simplified, enabling the model to avoid taking on excessive additional image correction work, thereby improving the recognition efficiency.

[0109] Subsequent machine model adjustment mainly refers to adjusting the internal parameters of the model through training and retraining to adapt to specific tasks or a constantly changing data environment. Its purposes include:

[0110] 1. Model feature extraction and generalization ability

[0111] Even after the image is preprocessed, the model itself still needs to learn how to extract effective features from the standardized image (such as the subtle color and shape differences between live and dead bacteria) for recognition.

[0112] The process of model adjustment is to enable the neural network to more precisely learn the differences between targets based on the existing features and improve the generalization ability.

[0113] 2. Adapt to long-term dynamic changes

[0114] As the mobile device ages over time, its hardware components will gradually deteriorate. Such changes often cannot be fully compensated simply through image preprocessing.

[0115] Subsequent model retraining can perform fine-tuning for specific devices and specific aging characteristics, enabling the model to continuously maintain high performance.

[0116] 3. Adapt to specific devices and user scenarios

[0117] The same preprocessing algorithm usually has difficulty comprehensively correcting for the individual differences of each specific device (such as local image blurring caused by lens aging, minor stains, or slight scratches).

[0118] By adding subsequent model adjustment, customized adjustments are made for the image features of specific mobile phones and specific users to make up for the detailed differences that cannot be overcome by a single image preprocessing.

[0119] All in all, preprocessing methods are generally suitable for a wide range of general scenarios, but overly general methods are difficult to fully adapt to the subtle differences of different model devices. Customizing a fine preprocessing process for each mobile phone is difficult to operate in practice because the device differences are too complex and diverse. The mobile phone aging process is not a linear and unified process but a dynamic and personalized change process. It is difficult to address the continuously subtle aging problems such as local sensor degradation and lens scratches only through preprocessing.

[0120] In consumer applications, to ensure a high-precision stability of the model for a long time to guarantee the user experience. It is difficult to maintain the accuracy only by preprocessing for a long time because the device conditions are constantly changing. Therefore, retraining and fine-tuning of the model are required later to maintain the accuracy. In practical applications, relying solely on preprocessing optimization often requires a large amount of manual adjustment or complex image correction processes, which are not only costly but also difficult to popularize to all user groups. While model fine-tuning can achieve precise adaptation with only a few standard image trainings, with both cost and efficiency advantages.

[0121] S3. Automatically locate the region of the bacterial counting plate based on image recognition to automatically focus on the counting grid region.

[0122] Specifically, in one embodiment, S3 includes the following steps S31 - S32.

[0123] S31. Input the preprocessed image into the trained lightweight segmentation network, and the network output is the segmentation region probability map.

[0124] In the solution of the present invention, the core recognition region of the microscopic image is the micro-grating region of the bacterial counting plate, which becomes the carrier for observing and recognizing bacterial particles after dropping the stained nursing solution sample. The bacterial counting plate is a conventional slide for quantitative detection of microorganisms, with regularly distributed counting gratings on its surface, and the grid regions between the gratings are of a fixed distance, usually used for counting the number of cells, bacteria or other particles per unit volume. The effective analysis of the microscopic image is limited to the interior of this counting plate region, especially the centrally symmetric counting grid region. The image content of this region contains both stained bacteria and regular linear boundaries as the positioning basis, which is the key region for realizing structured image analysis.

[0125] Due to possible certain offsets or angular errors during user shooting, there are differences in the position, size and orientation of the counting plate in the image. Simply relying on coordinate hard coding is difficult to adapt to actual changes. Therefore, in this step, a lightweight image segmentation network is introduced to automatically identify the region of the bacterial counting plate in the image. This network is a structured deep model that has been pre-trained on a standard image dataset, and its structure mostly adopts simplified architectures such as U-Net, DeepLabV3 or BiSeNet, with the advantages of strong feature extraction ability, small number of parameters, and being suitable for mobile deployment. The network input is the standardized microscopic image, and the output is a segmentation area probability map of the same size as the input image. The gray value of each pixel point in the map represents the probability value that the point belongs to the counting plate region, and the value range is [0,1]. The network structure generally includes two modules: an encoder and a decoder. The encoder is responsible for extracting multi-scale semantic features such as image texture and edges, and the decoder gradually restores the features to the original resolution and fuses the intermediate feature maps to achieve high-precision positioning.

[0126] During the construction of the segmentation network, microscopic images of the bacterial counting plate with masked annotations have been used as supervised training data. The annotation content is the pixel-level region mask, that is, which pixels belong to the grid region and which belong to the background region. In the training stage, the difference between the model output and the true mask is optimized through the cross-entropy loss function or the Dice coefficient loss, and finally the network is capable of accurately segmenting the counting plate region in any input image. For example, in an image in the training set, the annotator clearly marked the mask image corresponding to the edge of the counting grid region. The model learns a large number of such images to achieve the ability to distinguish grid lines and form an internal abstract representation of their spatial structure.

[0127] During the actual operation process, after the user completes image acquisition and preprocessing, the system inputs the image into this lightweight segmentation network, and the network quickly outputs the corresponding segmentation region probability map. The system then binarizes the probability map by setting a threshold (such as 0.5) to obtain a clear region boundary. This segmentation result can be used for boundary extraction before geometric correction, or for judging whether the shooting completely covers the effective region, or as a reference for subsequent image cropping and ROI extraction. Through this step, high-precision extraction of the bacterial counting plate region is achieved, making the entire image recognition process focused and targeted in space, avoiding redundant processing of the background region, and also providing a spatial boundary guarantee for subsequent bacterial identification and quantity statistics.

[0128] S32. Extract the outer boundary rectangle of the mask and crop the image.

[0129] In this step, the segmentation region probability map generated in the previous stage is first binarized. Usually, a fixed threshold (such as 0.5) or a method based on maximum entropy is used to set the pixels belonging to the bacterial counting plate region in the image to 1 and the background region to 0, forming a clear binary mask image.

[0130] After obtaining this binary mask, the system will apply morphological operations to further process the edge region to smooth the segmentation boundary, fill small holes, or remove isolated noise points, making the final boundary contour complete and continuous. Common operations include closing operations (first dilating and then eroding) to connect broken boundaries, or opening operations to remove misjudged small regions. After processing, only the connected region with a complete structure and the largest area on the mask is retained, which is the main part of the bacterial counting plate.

[0131] Next, the outer boundary of the mask region is identified through an edge detection algorithm (such as the Canny algorithm or a contour-based boundary extraction method), and then its minimum bounding rectangle is calculated. This rectangle not only provides the position and size information of the bacterial counting plate in the original image but also provides a clear boundary for the cropping operation. If the counting plate is detected to be rotated or tilted, the system will also calculate the rotation angle and correct the counting plate to a standard front-facing form through affine transformation or perspective transformation to ensure that the grid region in the output image is horizontal, with clear vertical and horizontal structures and consistent scales.

[0132] For example, in an original microscopic image, the bacterial counting plate is slightly rotated due to the shooting angle, and it appears as a tilted rectangle in the image. After probability map segmentation and boundary extraction, the system identifies the four corner points of its outer boundary and constructs a transformation matrix based on the perspective model to straighten the region into a standard rectangular region, and then precisely crops the original image according to this range. The cropped image is the ROI (Region of Interest) image for subsequent bacterial identification, and its content is a complete, centered, and clear bacterial counting grid region.

[0133] S4. Obtain a microscopic image and use a first machine model to identify bacterial particles; wherein, the microscopic image is obtained by microscopically photographing a portable device after a nursing solution and a bacterial suspension are mixed, stained with a stain for bacteria or fungi, and dropped into the micro-grid region of the bacterial counting plate.

[0134] Specifically, in one embodiment, S4 includes the following steps:

[0135] Input the preprocessed microscopic image into the first machine model, and the network output is the recognition result of the positions of bacterial particles.

[0136] In the previous steps, the microscopic image has been focused on the standard grid region of the bacterial counting plate. This region bears the stained sample formed after the nursing solution and the bacterial suspension are mixed. After being magnified and photographed through the microscopic lens, a large number of bacterial or fungal particles with regular shapes, similar sizes, and obvious color contrasts will appear in the image. These particles often have strong small target-like characteristics, that is, small sizes but dense distributions and complex background textures, so a specially designed target detection network is required for identification.

[0137] The first machine model is a bacterial body recognition model pre-trained on standardized microscopic image data. Its network structure generally adopts a lightweight object detection architecture, such as a small variant of YOLOv5-nano, EfficientDet-D0, or RetinaNet, with high detection speed and a small number of model parameters, and can be deployed on consumer-grade mobile terminals. The model extracts local texture and edge features of the image through convolutional layers, enhances the detection ability for particles of different particle sizes through a multi-scale feature fusion mechanism, and finally outputs a complete recognition result including the position box, confidence level, and feature vector of the bacterial body. The model has been trained on standard slide images and has the ability to generalize and learn visual features such as the morphology and staining reaction of the bacterial body.

[0138] During the execution process, the system sends the image to the input port of the model. The model quickly scans and extracts features from the image, identifies the position of each bacterial particle in the image, and returns the result in the form of structured data, such as the bounding box coordinates (x, y, w, h) and the corresponding recognition confidence score. For example, in an image containing stained bacterial bodies, the model identifies 78 bacterial particles, and the positioning of 76 of them coincides with the database labels, and the confidence levels are all higher than the set threshold (such as 0.7). The system then considers this recognition result valid and sends the batch of particle information to the next classification step.

[0139] S5. Classify and identify live and dead bacterial particles among the recognized ones to calculate the disinfection efficacy.

[0140] Specifically, in one embodiment, S5 includes the following steps S51 - S52.

[0141] S51. Identify live and dead bacteria based on the color and morphological characteristics after staining;

[0142] S52. Count the numbers of live and dead bacteria and calculate the proportions to calculate the disinfection efficacy.

[0143] In actual microscopic images, bacteria treated with a staining agent (such as thiazolyl blue or propidium iodide) will show different staining situations due to differences in cell membrane integrity and metabolic activity. Live bacteria usually appear dark blue or blue-violet because they can prevent the dye from entering or change the reduction state of the dye; dead bacteria, due to the destruction of the cell structure, have the dye directly penetrate into the cytoplasm and appear light-colored or nearly transparent.

[0144] In S51, the system calls a preset color and morphological feature analysis module to determine the living and dead states of each fragment of the bacterial cell particle image output by the first machine model. This module can be based on a shallow machine learning model (such as K-nearest neighbor or support vector machine), or a lightweight convolutional classifier can be used to model and classify features such as the color channel histogram, average RGB value, edge sharpness, and morphological compactness in the particle image. Taking the color average value and contrast of the image as an example, the living bacteria region usually has a low-saturation, high-contrast blue-violet region with clearly defined edges in the image, while the dead bacteria region has a grayish-white color and blurred edges. By comparing the labels of the living and dead bacteria samples in the training set through the feature extractor, a living and dead state distinction with relatively high accuracy can be achieved. The system judges each particle one by one and assigns it a label of "living" or "dead" to generate a complete classification result list.

[0145] After entering S52, the system performs quantity statistics based on the classification results generated in S51, accumulates the number of living bacteria and the number of dead bacteria respectively, and calculates the proportion of living bacteria or the killing rate of dead bacteria as a quantitative indicator of the disinfection efficacy after processing this sample. For example, if a total of 120 bacterial cell particles are identified in a microscopic image, and 95 of them are determined to be dead bacteria, the system outputs a killing rate of 95 / 120, that is, 79.2%. This statistical result can be used as the direct basis for evaluating the bactericidal performance of the nursing solution, and can be used for comparison with historical data, generating trend charts, or for pass / fail judgments.

[0146] The embodiment of the present application also discloses a method for image recognition of the disinfection efficacy of a contact lens care product based on a portable device, including a processor, and a program of the method for image recognition of the disinfection efficacy of a contact lens care product based on a portable device as described in any one of the above is run in the processor.

[0147] The embodiment of the present application also discloses a storage medium storing a program of the method for image recognition of the disinfection efficacy of a contact lens care product based on a portable device as described in any one of the above.

[0148] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. An image recognition method for the disinfection efficacy of a contact lens care product based on a portable device, characterized in that, Including the following steps: S1. Update the first machine model on the portable device according to the model and service life of the portable device; wherein, the first machine model is used to identify bacterial particles, and the portable device includes a smartphone camera and a macro / micro lens attachment; S2. Perform image acquisition and preprocessing to enable the output of an image with standardized color and brightness, reduced noise, and geometric alignment; S3. Automatically locate the area of the bacterial counting plate based on image recognition to automatically focus on the counting grid area; S4. Obtain a microscopic image and use the first machine model to identify bacterial particles; wherein, the microscopic image is obtained by microscopically photographing the portable device after the nursing solution and the bacteria-containing suspension are mixed, stained with a stain for bacteria or fungi, and dropped into the micro-grid area of the bacterial counting plate; S5. Classify and identify live and dead bacterial particles among the identified bacterial particles to calculate the disinfection efficacy.

2. The method for identifying the disinfection efficacy image of the contact lens care product based on a portable device according to claim 1, wherein, S1 includes the following steps: S11. Read the model and service life of the portable device; S12. Use the portable device to perform image acquisition and preprocessing on a number of standard slides; wherein, the standard slides correspond to pre-entered standard images in the database, and the coordinate positions of the bacteria, live / dead bacteria labels, color feature labels, and clarity labels of the standard images are pre-annotated; S13. Retrain the pre-trained first machine model stored on the portable device based on the images acquired by the portable device, the corresponding pre-entered standard images in the database, and the relevant annotations; S14. After the retraining of the machine model is completed, use a preset standard slide as the validation set to evaluate the performance difference between the retrained machine model and the machine model before retraining; if the performance of the retrained machine model is better than that of the machine model before retraining, then apply the retrained machine model to the portable device and deploy it; otherwise, continue to use the machine model before retraining.

3. The method for identifying the disinfection efficacy image of the contact lens care product based on the portable device according to claim 2, wherein The said S13 includes the following steps: S131. Retrieve the pre-trained first machine model stored on the portable device and freeze the front convolutional layer and the middle layer; S132. Add a number of adaptation layers after the frozen front convolutional layer and middle layer as device adaptation layers; wherein, the number and scale of the device adaptation layers are determined according to the mobile phone model and service life; S133. Individually train the newly added adaptation layers using the standard slide images uploaded by the user, and a relatively high learning rate is used for the individual training; S134. Unfreeze the basic network parameters and then perform a short-cycle overall fine-tuning using the same standard slide images, and a relatively low learning rate is used for the fine-tuning.

4. The method for identifying the disinfection efficacy image of the contact lens care product based on the portable device according to claim 1, wherein The said S2 includes the following steps: S21. Output user operation prompts to guide the user to align with the grid area of the bacterial counting plate; S22. Perform color correction and white balance on the image input; S23. Perform noise reduction and enhancement on the image input; S24. Perform geometric correction on the image input.

5. The method for identifying the disinfection efficacy image of a contact lens care product based on a portable device according to claim 1, characterized in that, S3 includes the following steps: S31. Input the preprocessed image into a trained lightweight segmentation network, and the network output is a segmentation region probability map; S32. Perform mask outer boundary rectangle extraction and image cropping.

6. The method for identifying the disinfection efficacy image of a contact lens care product based on a portable device according to claim 1, wherein The said S4 includes the following steps: The preprocessed microscopic image is input into the first machine model, and the network output is the recognition result of the positions of bacterial particles.

7. The method for identifying the disinfection efficacy image of the contact lens care product based on the portable device according to claim 1, wherein, S5 includes the following steps: S51. Identify live bacteria and dead bacteria based on the color and morphological characteristics after staining; S52. Count the numbers of live bacteria and dead bacteria, and the statistical results are used to estimate the disinfection efficacy index.

8. The method for identifying the disinfection efficacy image of the contact lens care product based on the portable device according to claim 1, characterized in that The pre-training steps of the first machine model include: S101. Obtain microscopic images of standardized slides taken by a specific model of portable device, where the microscopic images are multiple and cover different lighting conditions, different shooting angles, different sample concentrations, and staining concentrations; S102. Annotate and perform data augmentation on the microscopic images, where the annotation includes: the position and survival status of each bacterial / fungal particle, and the precise pixel mask of the grid area; S103. Perform initialization of the network structure and parameter settings, and divide the dataset of microscopic images into a training set, a validation set, and a test set; S104. Use the training set and the validation set to perform full-data training on the initialized model to obtain the first machine model for identifying bacterial particles.

9. An image recognition system for disinfecting efficacy of contact lens care products based on a portable device, characterized in that, It includes a processor, and a program of the method for identifying the disinfection efficacy of a contact lens care product based on a portable device as described in any one of claims 1-8 runs in the processor.

10. A storage medium, characterized in that, Store a program of the method for identifying the disinfection efficacy of a contact lens care product based on a portable device as described in any one of claims 1-8.

Images