Self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing

Abstract

This invention relates to the field of industrial data processing technology, specifically to a self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing. It collects two-dimensional data on lens contamination and the environment through imaging, spectral, and environmental sensors. This data is then processed through distortion correction, filtering and noise reduction, feature standardization, and multimodal fusion to generate a comprehensive situational data stream. A benefit-cost matrix is ​​constructed based on energy budget and optical performance thresholds. Cleaning priority and resource control rate are optimized to generate operation control system instructions. The system controls a first control unit and an acousto-optical tweezers array to remove contaminants and collects operation results and energy consumption data. Based on the situational-decision-execution-feedback data chain, optimized decision generation is achieved, enabling efficient, low-consumption, and adaptive cleaning of lenses in various contamination types and complex environments, improving optical performance stability and long-term operating efficiency.

Description

Technical Field

[0001] This invention relates to the field of industrial data processing technology, specifically to an anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing. Background Technology

[0002] In applications such as optical lenses, camera modules, and precision sensor windows, surface contamination has a significant impact on equipment performance, making the development of self-cleaning systems crucial.

[0003] Existing self-cleaning systems typically employ rudimentary program control mechanisms. For instance, their execution is often managed by a single, fixed control loop that triggers pre-defined cleaning subroutines based on simple hardware interrupts. This approach suffers from significant shortcomings in computational resource management: the system lacks a dynamic resource allocation mechanism; both processor time and cleaning energy consumption are allocated in a fixed manner, failing to intelligently schedule based on the complexity of real-time contamination data.

[0004] When processing data from multiple sensors such as imaging and spectroscopy, existing systems also lack efficient multi-task processing arrangements and inter-program communication mechanisms; the difficulty in achieving effective synchronization and fusion between various sensor data processing tasks makes it impossible for the main control program to obtain the optimal decision-making basis.

[0005] In summary, the bottleneck of the existing technology lies in its overly simple underlying program control arrangement, which is unable to perform concurrent management and adaptive resource scheduling for diverse sensing and cleaning tasks, thus limiting the overall operating efficiency and intelligence level of the self-cleaning system when facing complex polluted environments.

[0006] To address this, a self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing is proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing. Through adaptive control methods that combine multimodal sensing with benefit-cost trade-off decisions, the system can achieve automatic identification, optimized cleaning, and closed-loop adjustment of surface contamination on anti-fingerprint lenses.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] The self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing includes the following modules:

[0010] The perception module and the adaptive system collect two-dimensional data on lens contamination and the environment, extract features from the two-dimensional data on lens contamination and the environment, and generate a comprehensive situational data stream.

[0011] The decision-making module, the adaptive system receives the comprehensive situational data stream and constructs a benefit-cost matrix based on the preset energy budget and optical performance threshold as constraints. It then balances and solves the problem between cleaning priority, resource control rate and execution benefits, and generates operation control system instructions.

[0012] The execution module and adaptive system control the first control unit to clean the overall surface contaminants through the operation control system instructions, and control the acousto-optic tweezers array to remove particles; generate operation results and energy consumption data; the first control unit is used to control the speed and direction of airflow on the lens surface; the acousto-optic tweezers array controls sound waves and light beams to remove particles;

[0013] The feedback module receives work performance and energy consumption data from the adaptive system, compares them with the comprehensive situational data stream, and adjusts the work execution strategy during execution. It packages the situational-decision-execution-feedback data chain into an experience data package, optimizes the decision generation, and generates instructions for the future work control system.

[0014] Preferably, the specific steps for acquiring two-dimensional data on lens contamination and the environment, and extracting features from the two-dimensional data on lens contamination and the environment, include: acquiring high-resolution images of the lens surface obtained by an imaging sensor, and locating the contamination location using a control board; acquiring reflectance and transmission spectral data of the lens obtained by a spectral sensor, controlling the wavelength standard source for wavelength control, and performing baseline correction using a polynomial; the baseline correction includes: selecting background segments without obvious absorption peaks in the reflectance and transmission spectral data of the lens, controlling polynomial curve fitting and subtracting background signals; performing time alignment of the acquired multi-source data on imaging, spectral, and environmental data based on a unified time reference, and using bidirectional interpolation to fill sampling gaps to obtain two-dimensional data on lens contamination and the environment.

[0015] Preferably, the specific steps for generating the comprehensive situational data stream include: performing distortion correction and median filtering denoising on the lens contamination and environmental two-dimensional data; the distortion correction involves controlling the focal length, principal point position, and radial and tangential distortion coefficients obtained by the imaging sensor to map distorted pixel positions to distortion-free positions; the median filtering controls a sliding window to sort the pixel grayscale values ​​within each window and takes the median to replace the center pixel, and performs sliding window signal smoothing on the spectral data; the environmental data is processed using the 3σ principle to remove outliers and exponential smoothing to suppress transient fluctuations; the 3σ removal removes out-of-range measurements by adjusting the mean and standard deviation of the time period; the exponential smoothing refers to a method of weighting the current value with the previous smoothed value by an attenuation coefficient.

[0016] Based on the processed imaging data, pixel grayscale gradients are calculated and pollution locations are extracted through edge detection. Combined with connected component analysis, the area and boundary are calculated in groups according to pixel adjacency relationships. Based on the processed spectral data, the pollution type and concentration are identified by the control feature peak matching method. The imaging, spectral and environmental features are standardized, weighted and summed. A comprehensive pollution feature is generated by using a multimodal feature fusion algorithm, and a comprehensive situational data stream is output in time series.

[0017] Preferably, the specific steps for generating work control system instructions include:

[0018] By standardizing and weighting the multimodal characteristics of each polluted area, the cleaning priority is calculated; a benefit-cost matrix of polluted areas and available cleaning resources is established, and the action mode, intensity, timing and sequence of each execution module are determined through multi-objective optimization; the decision parameters are dynamically adjusted by combining historical execution data and environmental change predictions to generate operation control system instructions.

[0019] Preferably, the multimodal characteristics of each polluted area are standardized and weighted to calculate the cleaning priority; the specific steps for establishing a benefit-cost matrix of polluted areas and available cleaning resources include:

[0020] The cleaning benefit value is calculated for the contaminated area, and the cleaning priority is determined based on the pollution type, distribution density, contaminated area and pollutant characteristics. The energy consumption, cleaning efficiency and cleaning cost indicators generated by the execution module are matched with the cost of each module to generate a matrix based on the preset energy budget and optical performance threshold as constraints, and sorted according to the matrix weight. The value of the energy budget is determined by the lens size, and the optical performance threshold is determined by the light transmittance of the measured lens.

[0021] Preferably, the specific steps for generating operational performance and energy consumption data include:

[0022] After receiving instructions from the operation control system, the first control unit is controlled to blow away and loosen macroscopic pollutants on the lens surface according to the wind speed, wind direction and action time set in the instructions; according to the microscopic operation coordinates in the decision, the acoustic-optical tweezers array is controlled to remove individual particles; after the operation is completed, the surface condition after cleaning and the energy consumption data of each execution module are collected to generate operation results and energy consumption data.

[0023] Preferably, the specific steps for adjusting the job execution strategy during execution include:

[0024] Receive operational performance and energy consumption data from the execution modules; compare them with the comprehensive situational data stream to calculate pollution residue rate, cleaning efficiency, and energy consumption deviation; and adjust the operating mode, intensity, duration, and area of ​​each execution module based on the deviation.

[0025] Preferably, the specific steps for packaging the situation-decision-execution-feedback data chain into an experience data package, optimizing the decision-making module, and generating future operation control system instructions include:

[0026] The comprehensive situational data stream generated by the perception module, the operation control system instructions issued by the decision-making module, the operation effectiveness and energy consumption data fed back by the execution module, and the operation execution strategy during execution are organized into a situational-decision-execution-feedback data chain. Feature extraction and standardization processing are performed on the data chain to generate an experience data package. Based on the experience data package, the benefit-cost matrix of the polluted area and clean resources is updated, and based on the decision generation rules of the optimization decision-making module, the cleaning priority, resource allocation and action sequence are adjusted to predict future operation control system instructions.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. This invention uses imaging sensors, spectral sensors, and microenvironment sensors to collaboratively collect two-dimensional data on lens pollution and the environment. After multiple processing steps such as time alignment, distortion correction, median filtering, and outlier removal, combined with a multimodal feature fusion algorithm, a comprehensive situational data stream is generated. This enables synchronous and high-precision perception of pollution type, location, area, and environmental conditions, significantly improving the comprehensiveness and accuracy of pollution identification.

[0029] 2. In the decision-making stage, this invention introduces a benefit-cost matrix of contaminated areas and available clean resources, and uses cleaning priority, resource control rate and execution benefits as multi-objective optimization constraints for trade-off solution. It also combines historical execution data and environmental change predictions to dynamically adjust decision parameters, making the operation execution strategy more accurate and the energy consumption allocation more reasonable, avoiding the resource waste and insufficient cleaning caused by traditional fixed strategies.

[0030] 3. This invention establishes a closed-loop data system covering the entire process of situation, decision-making, execution, and feedback. It adjusts the operation execution strategy by comparing the operation results and energy consumption data with the comprehensive situation data stream, and packages the complete data chain into an experience data package to update the benefit-cost matrix and decision rules, thereby forming a continuous optimization and self-learning mechanism, which significantly improves the system's adaptability and long-term operating efficiency under different pollution types and environmental conditions. Attached Figure Description

[0031] Figure 1 The flowchart shows the self-cleaning adaptive control system for fingerprint-resistant lenses based on multimodal sensing.

[0032] Figure 2 This is a schematic diagram of the structure of an anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing.

[0033] Figure 3This is a diagram illustrating the interaction process of an anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Example 1:

[0036] This embodiment discloses a self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing, relating to the field of industrial data processing technology. This embodiment achieves adaptive optimization control by dynamically identifying and cleaning contaminants on the lens surface. The system includes a sensing module, a decision-making module, an execution module, and a feedback module, forming a closed-loop control system of situation-decision-execution-feedback. The system is suitable for scenarios such as customized home lenses, optical instruments, and smart display devices, and can achieve self-cleaning under different lighting, temperature, humidity, and air particle concentration conditions.

[0037] During the sensing phase, the system acquires images of the lens surface using a high-resolution imaging sensor, achieving a resolution of 5000×5000 pixels, and locates the contaminated area via a control panel. A spectral sensor collects the lens's reflectance and transmission spectra, covering a wavelength range of 400–1000 nm, controlled by a standard wavelength source. Polynomial baseline correction removes background signals; typical polynomial orders are 3 to 5. Environmental parameters such as temperature, humidity, and airborne particle concentration are acquired in real-time at a sampling frequency of 1 Hz. All data are aligned on a unified time reference, and bidirectional interpolation is used to fill sampling gaps.

[0038] By limiting the acquisition and preprocessing steps of raw data in the sensing module, the system ensures the original fidelity and synchronization consistency of the input data. Through specific means such as high-resolution imaging, spectral control, baseline correction, and time alignment of multi-source data, it solves the problems of possible errors, drift, and timing inconsistencies in the raw sensor signals from the source. This provides a high-quality and highly reliable raw data foundation for the subsequent generation of accurate situational data streams, which is the fundamental guarantee for the entire system to "see accurately".

[0039] The acquired data undergoes preprocessing. Imaging data is denoised using distortion correction and median filtering. Distortion correction parameters include focal length, principal point position, and radial and tangential distortion coefficients. The filtering window is typically 3×3 or 5×5 pixels. Spectral data is smoothed using a sliding window method, with a window length of 5 to 11 sampling points. Environmental data undergoes 3σ outlier removal and exponential smoothing to suppress transient fluctuations, with an exponential decay coefficient typically ranging from 0.2 to 0.5. After processing, the imaging data is used to extract contamination boundaries using Canny edge detection, and the contamination area and perimeter are calculated using connected component analysis. Spectral data is used to identify contamination types (such as grease, dust, or sweat) and concentrations using characteristic peak matching. All image, spectral, and environmental features are standardized, weighted, and fused to generate comprehensive pollution features. The specific method of weighting is as follows: a dynamic weighting method based on principal component analysis (PCA) is adopted. The weights of each modality data are dynamically adjusted by calculating the contribution of each modality data to the overall variance within the current time window, so as to ensure that the features most sensitive to the pollution situation receive higher weights. The comprehensive situation data stream is output in time series to form continuous and usable situation information.

[0040] This further defines the specific processing flow for transforming raw data into a comprehensive situational data stream, refining the multi-dimensional, noisy raw data into accurate and usable structured feature information. Through distortion correction, filtering and denoising, feature extraction (such as contamination boundaries, types, and concentrations), and multimodal feature fusion, it transforms scattered data points into a comprehensive and quantitative description of the lens contamination situation. This improves the system's accuracy and depth of understanding of contamination conditions, enabling subsequent decisions to be based on comprehensive and accurate information.

[0041] During the decision-making phase, the system standardizes the multimodal characteristics of each contaminated area and calculates the cleaning priority based on pollution type, distribution density, area, and pollutant characteristics. (This is for a lens area of ​​100cm².) 2 For small lenses within a certain range, the system can divide the contaminated area into 50 small grids, and calculate the benefit value of each grid separately; the system establishes a benefit-cost matrix between the contaminated area and available clean resources based on energy budget and optical performance threshold constraints;

[0042] The benefit values ​​in the matrix include cleaning efficiency, pollution reduction, and optical restoration, while the costs include energy consumption and operation time. The matrix is ​​solved by a multi-objective optimization algorithm to determine the action mode, intensity, duration, and sequence of each execution module. The decision parameters are dynamically adjusted by combining historical execution data and environmental change predictions.

[0043] To achieve precise quantitative decision-making, the system performs a multi-dimensional assessment of each identified contaminated area to calculate its "cleanup benefit value." This assessment integrates multiple factors: the system assigns a base hazard score based on the pollution type determined by spectral analysis, with higher hazard types receiving higher scores. The base score is then weighted and adjusted by combining the size of the contaminated area and the density of pollution distribution obtained from image analysis.

[0044] For example, a large area of ​​high-density contamination located in the optical center of a lens will ultimately yield a significantly higher "cleaning benefit" than scattered small particles in the edge areas; the system calculates the "execution cost" for each available cleaning action. This cost is primarily determined by the equipment power required to perform the action and the duration of operation; high-power, long-duration operation means higher costs.

[0045] The decision-making module constructs a virtual benefit-cost decision matrix. Within this matrix, the system matches each contaminated area with each possible cleaning action, evaluating the efficiency of each specific cleaning solution by calculating the ratio of benefit to cost. Once the matrix is ​​constructed, the system can clearly identify the cleaning solution combinations that offer the highest benefit with the lowest relative cost. The decision-making module prioritizes and executes these most efficient solutions, then processes other contaminated areas in descending order of efficiency, thereby generating the optimal overall job execution sequence and resource allocation strategy within given energy and performance constraints.

[0046] The core logical steps of the decision-making module in generating operation control system instructions are to standardize and weight the multimodal characteristics of each polluted area, calculate the cleaning priority, construct a benefit-cost matrix for multi-objective optimization, and dynamically adjust based on historical data to ensure that the final generated operation decision is the optimal solution under the constraints, thereby achieving a refined and rational allocation of clean resources (such as energy and time).

[0047] Preferably, a new implantable sensor is added to the sensing module. Specifically, the implantable sensor is a flexible piezoelectric thin film sensor array embedded between the antireflective coating of the lens and the substrate. This array is integrated synchronously with the coating process through physical vapor deposition (PVD) and can sense local micro-stress changes caused by coating wear or contaminant erosion, thereby quantifying the health status of the coating. The original strategy module is updated to a new strategy module and obtains the system's real-time energy reserves and the health status of the lens coating. The new module is based on a game theory algorithm model and does not only calculate the optimal solution for the current situation, but also solves for an optimal strategy sequence aimed at maximizing the long-term cumulative benefits of the system. The operation control system instructions are generated based on this optimal strategy sequence.

[0048] By creatively incorporating implanted sensors into the sensing module, the system has, for the first time, gained the ability to directly and in real-time monitor the health status of the lens itself (especially its functional coating). This differs from any indirect inference based on external observation, representing a high-precision and highly reliable "health check" method. It enables the system to detect its own "sub-healthy" state, such as microscopic wear or performance degradation of the coating, just as it detects external contamination, providing data input for subsequent proactive maintenance and repair.

[0049] This shift elevates the decision-making mechanism from "tactical optimization" to "strategic planning," achieving optimal operation throughout the system's entire lifecycle. The optimized strategy module is no longer a nearsighted calculator pursuing "local optima," but has evolved into a farsighted strategist. Based on a game theory algorithm model, it comprehensively weighs information from multiple dimensions, including the current situation, future risks, the system's energy reserves, and the lens's own health. Its goal is no longer simply to complete the immediate cleaning task, but to find the optimal strategy sequence that maximizes the system's long-term cumulative benefits. This revolutionary shift in decision-making paradigm enables the system to make high-level strategic decisions such as "delaying cleaning to conserve energy for future challenges" or "sacrificing some energy to proactively repair minor coating damage to extend overall lifespan," thereby fundamentally ensuring low operating costs, stable performance, and long lifespan throughout the system's entire lifecycle.

[0050] The execution module controls the first control unit and the acousto-optic tweezers array to complete the cleaning operation based on decision commands. The first control unit has an adjustable wind speed range of 0.5–3 m / s, and the wind direction can be finely adjusted according to the decision, achieving macroscopic blowing and loosening of contaminants on the lens surface. The acousto-optic tweezers array controls ultrasonic and visible laser beams in the range of 1MHz to 5MHz to remove particles, with a positioning accuracy of 0.01mm, suitable for cleaning individual dust or grease particles. After the operation is completed, the system collects images of the cleaned lens surface and energy consumption data of each execution module, forming operation results and energy consumption data, which are then uploaded to the feedback module.

[0051] By defining the tasks of the execution module and generating key feedback data, an efficient, layered operation method combining macro-level cleaning and micro-level removal is achieved, providing necessary feedback input for closed-loop control. The collaborative work of the first control unit and the acousto-optic tweezers array balances the breadth and precision of cleaning. By collecting operational effectiveness and energy consumption data after the operation, it provides the feedback module with the most direct and objective basis necessary for evaluating decision-making effectiveness and adjusting strategies.

[0052] The feedback module compares operational effectiveness and energy consumption data with the comprehensive situational data stream to calculate the residual pollution rate, cleaning efficiency, and energy consumption deviation. Based on the deviation, it dynamically adjusts the operating mode, intensity, duration, and area of ​​each execution module to achieve adaptive optimization. For example, when the residual pollution rate exceeds 5%, the system can increase the number of times the acousto-optic tweezers array operates or extend the airflow blowing time. Conversely, when energy consumption exceeds the budget, the system can automatically reduce the wind speed or decrease localized operations.

[0053] The strategy of limiting the feedback module to make real-time adjustments gives the system tactical-level real-time adaptive correction capabilities during a single operation. By comparing the execution feedback data with the original situation data, calculating the deviation and dynamically adjusting the execution parameters accordingly, the system can cope with unexpected situations that occur during execution to ensure the success rate and robustness of a single cleaning task.

[0054] All situation-decision-execution-feedback data chains are organized and feature extracted to form an experience data package, which is used to update the benefit-cost matrix and optimize cleaning priorities and resource allocation, thereby enabling prediction and optimization of future operations.

[0055] The learning mechanism that limits the feedback module to long-term optimization endows the system with strategic-level self-learning and evolution capabilities, ensuring the optimization of its long-term operating efficiency. By packaging the complete situation-decision-execution-feedback data chain and using it for the benefit-cost matrix and decision rules, the system can accumulate experience from every success and failure. This enables it to continuously adapt to environmental changes, the evolution of pollution types, and even the aging of its own components, achieving continuous iteration and improvement of performance.

[0056] Preferably, the feedback model can also transmit the situation-decision-execution-feedback experience data package to the model evolution unit; the model evolution unit has a built-in online reinforcement learning algorithm, which extracts the work performance and energy consumption data from the experience data package and converts them into a reward or penalty signal according to preset rules; the model evolution unit rewards efficient cleaning effect and deducts points for energy consumption and execution time based on the reward or penalty signal; this reward and punishment mechanism will guide the system to learn how to make the best trade-off between cleanliness, energy consumption and efficiency; and perform update operations on the policy network of the decision algorithm model built into the decision module to adjust the policy tendency of the model to generate decisions when facing similar situations in the future.

[0057] This solution upgrades the decision-making model. Unlike traditional feedback mechanisms that can only fine-tune within a preset parameter framework, this solution introduces a model evolution unit. Using complete empirical data as a training set, it continuously retrains and updates the decision-making strategy network, enabling the system to self-evolve. Its significance lies not only in optimizing behavior but also in reshaping and even surpassing the original decision-making logic to cope with unknown environments and operating conditions. When the system encounters pollution types or extreme conditions unforeseen by the designers, the online reinforcement learning mechanism, through a "trial-and-error-feedback-learning" cycle, can autonomously explore and generate new, effective strategies, demonstrating strong robustness in the face of uncertainty. Every piece of empirical data accumulated during system operation becomes the driving force for evolution, making it "smarter with use," and continuously improving performance and efficiency over time. More importantly, this mechanism significantly extends the system's lifespan and possesses high scalability, enabling seamless integration of new sensors or execution modules in the future, achieving synchronous evolution of hardware and intelligence.

[0058] Through the above process, this embodiment achieves multimodal data fusion, dynamic adaptive control, and precise particle cleaning, and continuously improves cleaning efficiency and energy control through closed-loop optimization. In practical applications, the system can significantly improve lens cleaning efficiency while reducing energy consumption, significantly improve the anti-fingerprint performance of the lens surface, and maintain good optical transmittance and stability. It is suitable for scenarios such as smart display devices, optical instruments, and customized home lenses.

[0059] Example 2:

[0060] In this embodiment, the anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing is applied to the lens self-cleaning control of various types of optical devices, involving the field of industrial data processing technology; including head-mounted display devices, high-precision microscopic imaging instruments, and outdoor optical observation equipment, etc.

[0061] This embodiment enhances and supplements aspects such as sensor configuration, decision matrix construction, execution mode adjustment, and feedback optimization, thereby improving the system's operational stability and cleaning accuracy in complex environments.

[0062] The sensing module adds attitude sensors and vibration sensors to the existing imaging sensors, spectral sensors and microenvironment sensors. It monitors the real-time tilt angle, rotation state and degree of external mechanical disturbance of the equipment where the lens is located to determine the relationship between pollution distribution and gravity direction, and identifies external factors that cause secondary adhesion of pollutants.

[0063] New sensor data, along with imaging, spectral, and environmental information, are synchronously calculated using a preset interpolation algorithm under a unified time reference to generate the corresponding data value for that target moment, forming a time-consistent extended integrated situational data stream.

[0064] This solution enhances the system's perception dimension and cognitive depth in complex dynamic environments. By introducing attitude and vibration sensors, the system's perception capability has leaped from "two-dimensional" contamination analysis limited to the lens surface to a "three-dimensional" dynamic understanding of the physical environment in which the equipment is located.

[0065] This solution not only identifies pollution categories but also understands the physical state of the pollution, enabling deeper judgment and prediction of pollutant behavior. It enhances the predictability and proactive defense capabilities of the decision-making system. Traditional systems can only passively clean up after pollutants appear, while this solution can identify high-risk environments by monitoring the attitude and disturbances of the equipment. This allows the system to shift from post-event remediation to pre-event warning and proactive defense, such as adjusting cleaning strategies in advance or entering a prepared state, intervening before pollutants solidify and adhere, thus reducing cleaning difficulty and energy consumption.

[0066] This approach fundamentally ensures the accuracy and reliability of multi-source heterogeneous data fusion. By forcing all sensor data to synchronize under a unified time reference and controlling the interpolation algorithm to fill sampling gaps, this scheme solves the core technical challenge of data asynchrony caused by differences in sampling frequency and operating timing among different sensors. It ensures that at the moment of decision-making, the system relies on a time-aligned, distortion-free situational snapshot, providing a solid data foundation for the effectiveness and accuracy of all subsequent advanced decision-making algorithms and significantly enhancing the robustness of the entire system.

[0067] When constructing the benefit-cost matrix in the decision-making module, a scenario-adaptive weighting factor is introduced to dynamically change the cleaning task priority for different types of devices. For head-mounted displays, the core visible area is marked based on the user's average field of view, and its benefit value is increased to ensure that key visual areas are cleaned first.

[0068] For microscopic imaging equipment, the highest benefit value is set for the central area of ​​the optical path, and the priority is adjusted according to the magnification. For outdoor observation equipment, wind speed and dust concentration are used to calculate the redeposition risk score. When the threshold is exceeded, the cleaning cycle is automatically shortened and a high-power airflow mode is activated.

[0069] In terms of the execution module, the first control unit adopts a dynamic jet adjustment mechanism to pre-blow the contaminated area with low-speed, wide-range airflow, causing loose particles to detach from the surface. According to the rate of decrease in particle concentration, it switches to high-speed directional airflow to remove firmly attached particles. The acousto-optic tweezers array automatically adjusts the sound wave frequency and beam focusing diameter based on the particle size distribution obtained from spectral analysis, realizing customized capture and removal of particles of different sizes, avoiding secondary interference to the cleaned area. The feedback module, while collecting data on operation effectiveness and energy consumption, adds a re-contamination trend prediction function. By combining parameters such as dust concentration, wind speed, and vibration frequency in the short period after cleaning with historical redeposition data, a prediction model is established, outputting the future pollution development curve and the recommended time for the next cleaning.

[0070] Add a recontamination risk parameter field to the situation-decision-execution-feedback data chain, and combine it with other operational data to form an experience data package. Optimize the decision generation rules to enable the scenario model to continuously and adaptively update over time, thereby forming the best cleaning strategy for a specific environment in the long run.

[0071] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A self-cleaning adaptive control system for anti-fingerprint lenses based on multimodal sensing, characterized in that, Includes the following modules: The perception module and the adaptive system collect two-dimensional data on lens contamination and the environment, extract features from the two-dimensional data on lens contamination and the environment, and generate a comprehensive situational data stream. The decision-making module receives comprehensive situational data streams from the adaptive system. It calculates clean-up benefits by standardizing and weighting the multimodal characteristics of each polluted area. Based on preset energy budgets and optical performance thresholds as constraints, it matches regional benefits with the energy efficiency and cost indicators of the execution module, calculates clean-up priorities, establishes a benefit-cost matrix of polluted areas and available clean resources, and sorts them according to matrix weights. It then balances clean-up priorities, resource control rates, and execution benefits to generate operation control system instructions. The revenue-cost matrix introduces a scene adaptation weight factor. For head-mounted display devices, the core visible area is marked based on the user's average field of view and its revenue value is increased. For microscopic imaging devices, the highest revenue value is set for the central area of ​​the optical path, and the priority is adjusted according to the magnification. The execution module, an adaptive system, controls the first control unit to clean the overall surface contaminants through the operation control system commands, and controls the acousto-optic tweezers array to remove particles; it generates operation results and energy consumption data; the control unit is used to control the speed and direction of airflow on the lens surface; the acousto-optic tweezers array controls sound waves and light beams to remove particles; The feedback module receives operational performance and energy consumption data from the adaptive system, compares them with the comprehensive situational data stream, and adjusts the operational control system commands during execution. It packages the situational-decision-execution-feedback data chain into an experience data package, optimizes the decision generation, and generates future operational control system commands.

2. The anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing according to claim 1, characterized in that: The specific steps for collecting two-dimensional data on lens contamination and the environment, and extracting features from this data, include: acquiring high-resolution images of the lens surface from an imaging sensor and locating the contamination location using a control panel; acquiring reflectance and transmission spectral data of the lens from a spectral sensor and controlling the wavelength standard source for wavelength control, and performing baseline correction using a polynomial; the baseline correction includes: selecting background segments without obvious absorption peaks in the reflectance and transmission spectral data of the lens, controlling polynomial curve fitting and subtracting the background signal; performing time alignment of the acquired multi-source data on the imaging, spectral, and environmental data based on a unified time reference, and using bidirectional interpolation to fill the sampling gaps to obtain two-dimensional data on lens contamination and the environment.

3. The anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing according to claim 1, characterized in that: The specific steps for generating the comprehensive situational data stream include: performing distortion correction and median filtering denoising on the lens contamination and environmental two-dimensional data. The distortion correction involves controlling the focal length, principal point position, and radial and tangential distortion coefficients obtained by the imaging sensor to map distorted pixel positions to distortion-free positions. The median filtering controls a sliding window to sort the pixel grayscale values ​​within each window and takes the median to replace the center pixel, performing sliding window signal smoothing on the spectral data. The environmental data is processed using the 3σ principle to remove outliers and exponential smoothing to suppress transient fluctuations. The 3σ removal removes out-of-range measurements by adjusting the mean and standard deviation of the time period. The exponential smoothing refers to a method of weighting the current value with the previous smoothed value by an attenuation coefficient. Based on the processed imaging data, pixel grayscale gradients are calculated and pollution locations are extracted through edge detection. Combined with connected component analysis, the area and boundary are calculated in groups according to pixel adjacency relationships. Based on the processed spectral data, the pollution type and concentration are identified by the control feature peak matching method. The imaging, spectral and environmental features are standardized, weighted and summed. A comprehensive pollution feature is generated by using a multimodal feature fusion algorithm, and a comprehensive situational data stream is output in time series.

4. The anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing according to claim 1, characterized in that: The specific steps for generating operational performance and energy consumption data include: After receiving instructions from the operation control system, the first control unit is controlled to blow away and loosen macroscopic pollutants on the lens surface according to the wind speed, wind direction and action time set in the instructions; according to the microscopic operation coordinates in the decision, the acoustic-optical tweezers array is controlled to remove individual particles; after the operation is completed, the surface condition after cleaning and the energy consumption data of each execution module are collected to generate operation results and energy consumption data.

5. The anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing according to claim 1, characterized in that: The specific steps of adjusting the job execution strategy during execution include: Receive operational performance and energy consumption data from the execution modules; compare them with the comprehensive situational data stream to calculate pollution residue rate, cleaning efficiency, and energy consumption deviation; and adjust the operating mode, intensity, duration, and area of ​​each execution module based on the deviation.

6. The anti-fingerprint lens self-cleaning adaptive control system based on multimodal sensing according to claim 1, characterized in that: The specific steps for packaging the situation-decision-execution-feedback data chain into an experience data package, optimizing the decision-making module, and generating future operation control system instructions include: The comprehensive situational data stream generated by the perception module, the operation control system instructions issued by the decision-making module, the operation effectiveness and energy consumption data fed back by the execution module, and the operation execution strategy during execution are organized into a situational-decision-execution-feedback data chain. Feature extraction and standardization processing are performed on the data chain to generate an experience data package. Based on the experience data package, the benefit-cost matrix of the polluted area and clean resources is updated, and based on the decision generation rules of the optimization decision-making module, the cleaning priority, resource allocation and action sequence are adjusted to predict future operation control system instructions.

Images