Image correction method and device of intelligent device, storage medium and electronic device

Abstract

This application provides an image correction method and apparatus, storage medium, and electronic device for smart devices. The method includes: responding to an image acquisition command, acquiring an image to be corrected and the device's operating status; being able to identify the material properties of a cover plate in real time or periodically and determine the current operating status; the material properties of the cover plate dynamically change with the physical degradation of the cover plate; selecting a target correction strategy that matches the current operating status from multiple correction strategies; fully considering the mutual influence between the device's operating status and the material properties of the cover plate to ensure that the target correction strategy can more accurately address the challenge of image quality degradation caused by the physical degradation of the cover plate; performing image correction on the image to be corrected to generate a quality-optimized corrected image; and using the corrected image as input for image recognition to improve the accuracy of biometric recognition, thereby significantly improving the image recognition success rate of smart devices under cover plates of different materials.

Description

Technical Field

[0001] This application relates to the technical field of image recognition processing, and more specifically, to an image correction method and apparatus, storage medium and electronic device for a smart device. Background Technology

[0002] In many devices that rely on image recognition technology, sensors or cameras often need to capture images through a specific cover plate or medium. While this cover plate or medium enhances the aesthetics, factors such as uneven ink layer distribution, thickness variations, material aging, and surface contamination can introduce image degradation phenomena such as uneven local illumination, decreased contrast, and color shift, thereby reducing the success rate of image recognition. Summary of the Invention

[0003] This application provides an image correction method and apparatus, storage medium and electronic device for smart devices, to at least solve the technical problem in the related art where the image recognition success rate decreases due to image degradation caused by the cover material.

[0004] According to one aspect of the embodiments of this application, an image correction method for a smart device is provided, comprising: in response to an image acquisition command, acquiring an image to be corrected of the smart device and the device operating state of the smart device; determining the current operating state of the smart device based on the device operating state and the material properties of a cover plate; the material properties of the cover plate are the material properties of the cover plate of the smart device; the material properties of the cover plate are detected in response to the image acquisition command or periodically detected according to a preset acquisition cycle; the material properties of the cover plate dynamically change with the physical degradation of the cover plate; determining a target correction strategy matching the current operating state from a variety of correction strategies, and using the target correction strategy to perform image correction on the image to be corrected to obtain a corrected image, wherein the corrected image is used for image recognition.

[0005] According to another aspect of the embodiments of this application, an image correction apparatus for a smart device is also provided, comprising: an acquisition unit, configured to acquire an image to be corrected of the smart device and the device operating state of the smart device in response to an image acquisition command; a determination unit, configured to determine the current operating state of the smart device based on the device operating state and the material properties of a cover plate; the material properties of the cover plate are the material properties of the cover plate of the smart device; the material properties of the cover plate are detected in response to the image acquisition command or periodically detected according to a preset acquisition cycle; the material properties of the cover plate dynamically change with the physical degradation of the cover plate; and an execution unit, configured to determine a target correction strategy matching the current operating state from a variety of correction strategies, and use the target correction strategy to perform image correction on the image to be corrected to obtain a corrected image, wherein the corrected image is used for image recognition.

[0006] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed by a processor.

[0007] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in any of the method embodiments described above.

[0008] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0009] This application, by responding to image acquisition commands, obtains the image to be corrected and the device's operating status from the smart device. It can detect and identify the cover material properties of the smart device in real time or periodically, determining the current operating status of the smart device. The cover material properties dynamically change with the physical degradation of the smart device's cover. Furthermore, it selects a target correction strategy matching the current operating status from multiple correction strategies, fully considering the mutual influence between the device's operating status and the cover material properties. This ensures that the selected target correction strategy can more accurately address the challenge of image quality degradation caused by cover physical degradation. The target correction strategy is applied to the image to be corrected, performing image correction to generate a quality-optimized corrected image, thereby improving the input image quality for the image recognition process. The corrected image serves as the input for the image recognition function, enhancing the accuracy and robustness of biometric recognition. This effectively overcomes the poor image quality caused by changes in cover material properties, significantly improving the image recognition success rate of the smart device under different cover material conditions. Therefore, it can solve the technical problem in related technologies where image degradation caused by cover material leads to a decrease in image recognition success rate. Attached Figure Description

[0010] Figure 1 This is a schematic diagram illustrating an application scenario of an image correction method for a smart device according to an embodiment of this application;

[0011] Figure 2 This is a schematic flowchart of an optional image correction method for a smart device according to an embodiment of this application;

[0012] Figure 3 This is a flowchart of an optional image correction method according to an embodiment of this application;

[0013] Figure 4 This is an optional system architecture block diagram according to an embodiment of this application;

[0014] Figure 5 This is a structural block diagram of an optional image correction device for a smart device according to an embodiment of this application;

[0015] Figure 6 This is a computer system architecture block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] According to one aspect of the embodiments of this application, an image correction method for a smart device is provided. Optionally, in this embodiment, the above-described image correction method for a smart device may be applied, but is not limited to, to applications such as... Figure 1 The hardware environment shown includes a smart device 102 and a server 104. The server 104 can be connected to the smart device 102 via a network and can be used to provide services (e.g., application services, etc.) to the smart device 102 or clients installed on the smart device 102. A database can be set up on or independently of the server 104 to provide data storage services for the server 104.

[0019] The aforementioned network may include, but is not limited to, at least one of the following: wired network and wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), and local area network (LAN). The aforementioned wireless network may include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI) and Bluetooth. Smart device 102 may be, but is not limited to, a personal computer (PC), mobile phone, tablet computer, smart door lock, smart camera, smart wearable device, etc. Server 104 may be, but is not limited to, a cloud server, server cluster, or other server types.

[0020] The image correction method for smart devices in this embodiment can be executed by server 104, smart device 102, or jointly by server 104 and smart device 102. Alternatively, the image correction method can be executed by a client installed on smart device 102.

[0021] Taking the image correction method of the smart device in this embodiment as an example, which is executed by the smart device 102, Figure 2 This is a schematic flowchart of an optional image correction method for a smart device according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps S202 to S206.

[0022] Step S202: In response to the image acquisition command, acquire the image to be corrected from the smart device and the device operating status of the smart device.

[0023] Step S204: Determine the current operating status of the smart device based on the device's operating status and the cover material properties; the cover material properties are the material properties of the smart device's cover; the cover material properties are obtained by detection in response to image acquisition commands or by periodic detection according to a preset acquisition cycle; the cover material properties change dynamically with the physical degradation of the cover.

[0024] Step S206: Determine the target correction strategy that matches the current working condition from multiple correction strategies, and use the target correction strategy to perform image correction on the image to be corrected to obtain the corrected image, wherein the corrected image is used for image recognition.

[0025] The image correction method for smart devices in this embodiment can be applied to the field of image recognition processing technology, specifically to scenarios where smart devices recognize and correct images. This method can be applied to smart door locks, enabling them to be opened via image recognition; it can also be applied to mobile terminals, unlocking them via image recognition; it can be applied to biometric attendance systems in enterprises or institutions, enabling employee attendance tracking via image recognition; it can be applied to driver monitoring systems for autonomous vehicles, monitoring driver attention via image recognition; and it can be applied to customer identification in smart retail systems, where image recognition is used for customer identification or behavioral analysis.

[0026] In many smart devices that rely on pre-image recognition technology, sensors or cameras often need to capture images through a specific cover plate or medium. While the cover plate or medium can enhance the aesthetics, factors such as uneven ink layer distribution, thickness variations, material aging, and surface contamination on the cover plate or medium can introduce image degradation such as uneven local lighting, low contrast, and color shift, which in turn leads to a decrease in the success rate of image recognition.

[0027] Taking smart door locks as an example, the following methods are used in related technologies to address the issue of camera performance degradation in smart devices: the currently used face or palm vein recognition module captures the image; if the image features are unclear, it is determined whether the image is obstructed by the lens (such as dirt or water droplets); if the obstruction affects the current recognition path, the system switches to another available biometric module (such as switching from face recognition to palm vein recognition); the new modality is used to complete identity verification and unlock the door. However, the aforementioned technologies only passively respond to local occlusion and cannot cope with global optical degradation. This is because they rely on unclear image features as a trigger condition and detect the presence of local dirt or water droplets that obstruct the view. As a result, the processing mechanism is only activated when the image has been severely degraded (such as local blurring or black spots). For overall image degradation (such as low illumination, color shift, or decreased contrast caused by dark glass covers), since the image still presents a "complete face," the system will not determine it as "unclear features" and therefore will not initiate any compensation or switching process. This results in the face recognition model being called even when the input image quality is poor but there is no obvious occlusion, leading to an increased false recognition rate or failure of liveness detection. The aforementioned technologies do not repair or enhance the image itself, relying solely on modality switching. This is because no image restoration or correction mechanism is introduced in these technologies. Therefore, if recognition fails, the current modality must be abandoned, and other recognition methods (such as palm vein recognition) must be used instead. This necessitates the use of multiple independent biometric hardware (such as simultaneously supporting face recognition and palm vein recognition), increasing system complexity and cost. On devices that only support a single recognition modality (such as face recognition only), this solution cannot provide any fault tolerance. Furthermore, in the aforementioned technologies, the facial recognition module of smart locks is typically integrated into the surface of the lock panel. Its imaging path needs to penetrate a cover plate with decorative ink or colored glass. While this cover plate enhances the aesthetics, it introduces the following optical interferences: selective absorption (different wavelengths of light are absorbed to varying degrees (e.g., dark gray ink strongly absorbs blue light and some near-infrared light)); scattering effect (frosted or textured glass causes light diffusion, reducing image sharpness); and reflection interference (surface reflection of ambient light, leading to local overexposure or glare). Therefore, smart devices cannot perceive material differences and lack pre-compensation capabilities. This is because the relevant technologies do not consider the impact of the physical material of the lock cover plate (e.g., transparent glass, dark ink screen printing layer, metal decorative frame) on imaging quality, causing the system to be unable to distinguish between "temporary contamination" and "permanent low light transmittance design." For products with dark cover plates installed at the factory, each recognition faces a low signal-to-noise ratio input, but the system is unaware of this and offers no compensation. This results in a long-term sub-optimal recognition state, degrading the user experience and increasing security risks (e.g., vulnerability to print attacks).

[0028] To at least partially solve the aforementioned technical problems, this embodiment improves the success rate of face recognition under low-transmittance, high-absorption cover plates without relying on multimodal redundant hardware through active triggering correction. By using material perception and active image correction mechanisms, it avoids dependence on backup modalities such as palm veins. Before the image deteriorates to the point of "unclear features," it identifies and compensates for optical degradation caused by the cover plate material in advance. It introduces a multispectral perception and physical modeling paradigm to restore image quality before recognition. Furthermore, it dynamically selects the optimal image correction path based on different cover plate material types (such as transparent / dark / metallic), balancing power consumption and accuracy. In this embodiment, in response to the image acquisition command, the image to be corrected and the device's operating status of the smart device are acquired. The cover material properties of the smart device can be detected and identified in real time or periodically to determine the current operating status of the smart device. The cover material properties dynamically change with the physical degradation of the smart device's cover. Furthermore, a target correction strategy matching the current operating status is selected from multiple correction strategies, fully considering the mutual influence between the device's operating status and the cover material properties. This ensures that the selected target correction strategy can more accurately address the challenge of image quality degradation caused by cover physical degradation. The target correction strategy is applied to the image to be corrected, performing image correction to generate a quality-optimized corrected image, thereby improving the input image quality of the image recognition process. The corrected image serves as the input for the image recognition function, enhancing the accuracy and robustness of biometric recognition. This effectively overcomes the poor image quality caused by changes in cover material properties, significantly improving the image recognition success rate of the smart device under different cover material conditions. Therefore, it can solve the technical problem in related technologies where image degradation caused by cover material leads to a decrease in image recognition success rate.

[0029] In this embodiment, the image acquisition command refers to the command in the smart device used to trigger the acquisition of the image to be corrected. The image to be corrected is the image that the smart device needs to correct. It can be understood that the image to be corrected can be the original image captured by the smart device when it starts working in response to the image acquisition command, that is, an image that has not undergone any processing. For example, in a biometric identification scenario, the image to be corrected is the original image of biometric features such as face or fingerprint captured by the camera or sensor of the smart device when the user unlocks or logs in.

[0030] The device operating status of a smart device refers to the operating state of the smart device in response to an image acquisition command. Optionally, the device operating status of a smart device includes various operating parameters and environmental information. For example, the device operating status of a smart device may include the device's battery level, system mode (such as low power mode), device temperature, ambient light intensity, etc.

[0031] The cover material properties refer to the physical material properties of the cover plate in front of the camera of a smart device. Optionally, the cover plate can be the front panel of a smart lock, typically made of glass or acrylic, with a decorative ink or film coated on the surface to shield internal components. Its light transmission characteristics affect image quality. Cover material properties may include, but are not limited to, light transmittance, surface roughness, absorption coefficient, scattering effect, ink thickness and distribution, etc. For example, cover material properties describe the physical response characteristics of the cover plate to light, including primary color c^, transmittance spectral function τ^(λ), surface roughness r^, reflectance ρ^, and absorption coefficient α^.

[0032] It should be noted that the material properties of the cover plate directly affect the quality of images acquired and captured by smart devices. Over long-term use, the cover plate material may change due to wear, contamination, aging, and other factors, leading to global optical interference and manifesting as physical degradation of image quality. Physical degradation refers to the suboptimal optical performance of the transparent or semi-transparent medium (such as a glass cover plate) placed in the imaging path due to manufacturing processes, material characteristics, or long-term use. Specifically, this can include uneven ink layer distribution, thickness variations, material aging, and surface contamination. These factors can introduce image degradation phenomena such as uneven local illumination, decreased contrast, and color shift.

[0033] It is understandable that the material properties of the cover plate change dynamically according to the degree of physical degradation of the cover plate. The material properties of the cover plate are not a fixed value, but a changing value. Therefore, it is necessary to detect the material properties of the cover plate in real time or periodically in response to the image acquisition command.

[0034] Optionally, a cover material sensing module can be set in the smart device. This module can detect the cover material properties once in response to each image acquisition command, or it can automatically detect the cover material properties at preset time intervals (e.g., daily, weekly, or each time the device is started), thus obtaining the cover material properties over a period of time. In this way, compared to the passive triggering in related technologies, this embodiment avoids recognition failure by actively sensing the material and compensating in advance.

[0035] The current operating status is the state of the working environment and conditions of the smart device in response to the image acquisition command. For example, the current operating status could be that the smart device is in a low-power operation state, or it could be that the smart device is in a stable operation state, or it could be that the smart device is in an outdoor operating condition, or it could be that the smart device is in a complex operating condition.

[0036] Optionally, if the current battery level of the smart device is below a specified threshold (e.g., 20%) and the cover material is a high-absorption material (e.g., dark glass), then the current operating state of the smart device is determined to be a low-power, high-interference state; or, if the current battery level of the smart device is above 50% and the cover material is transparent glass, then the current operating state of the smart device is determined to be a high-energy-efficiency, low-interference state; or, if the user attempts face recognition continuously under normal lighting conditions but fails more than twice, and the smart device's status information shows sufficient battery power (greater than 75%), and the cover material is dark glass with a light transmittance as low as 30% and a significant local ink layer thickness, then the current operating state of the smart device is determined to be a high-performance demand state.

[0037] In this embodiment, the smart device pre-stores multiple correction strategies, each corresponding to a different operating condition of the smart device; that is, one correction strategy corresponds to each operating condition. One of the multiple correction strategies refers to a pre-set correction strategy based on the operating condition of the smart device. These strategies include those for correcting the image to be corrected under different operating conditions, used to restore the quality of the image and improve the accuracy of image recognition. Optionally, the correction strategy can be dynamically selected based on the current operating condition of the smart device (such as battery level or cover material properties) to achieve the best image correction effect and resource utilization. For example, the correction strategy may include fast image correction based on lookup tables (LUTs), high-precision neural network model correction, or a correction process that adaptively adjusts parameters according to a specific scenario. It may also be a correction based on a physical model, a correction strategy based on machine learning, etc.

[0038] The target correction strategy refers to the intelligent device determining a correction strategy that matches the current operating condition from a variety of preset correction strategies. After determining the target correction strategy, it is used to perform image correction on the image to be corrected. Image correction refers to improving image quality by adjusting image attributes. Optionally, image correction may include, but is not limited to, white balance adjustment, tone equalization, color mapping, ambient lighting adaptation adjustment, saturation and contrast enhancement, etc. The purpose of image correction is to make the image colors closer to the colors in the real environment, thereby improving the image recognition success rate.

[0039] Optionally, when the battery power of the smart device is less than 20%, and the system detects face recognition three times consecutively, and the cover is dark glass with a light transmittance of 20%, the current working condition is determined to be a low battery working condition, and a target correction strategy matching the current working condition is determined.

[0040] It should be noted that the causes and degrees of image degradation vary among smart devices under different usage environments and physical conditions. Determining the current operating state of the smart device based on its operating status and cover material properties is to select the most suitable correction strategy for that current state. This ensures that the target correction strategy for the image can more accurately adapt to the current device conditions and cover material properties, thereby achieving reasonable resource allocation, optimizing image quality, and improving the accuracy of biometric recognition. Furthermore, different operating states require different correction strategies to overcome different types of image degradation problems. By flexibly switching these correction strategies, a balance between high performance and power consumption can be maintained under various operating conditions, thus enabling the smart device to operate stably and achieve efficient recognition in complex environments.

[0041] After image correction is performed on the image to be corrected, a corrected image is obtained. The corrected image refers to the image with improved quality obtained after the smart device applies the target correction strategy to the original image to be corrected. Optionally, the corrected image shows significant improvements in color consistency, contrast, and detail sharpness.

[0042] Optionally, assuming the smart device is currently operating outdoors with varying lighting conditions and the cover material is transparent but exhibits physical degradation due to lighting changes, the target correction strategy is determined to be an ambient light sensor combined with intelligent exposure and color dynamic correction. This automatically adjusts image exposure time and color balance based on light intensity to adapt to constantly changing lighting conditions. The smart device monitors ambient light intensity in real time using its built-in ambient light sensor and incorporates this intensity into the image correction process. Based on the light intensity, the camera's exposure time and gain are dynamically adjusted to prevent overexposure in high-light environments and ensure sufficient light intake in low-light environments. Then, based on the light intensity information, dynamic range compression technology and color correction filters are used to adjust the image's color saturation, contrast, and white balance, ensuring color consistency and realism under different lighting conditions. Through this dynamic correction, the corrected image generated by the device is more accurate in color and exposure, maintaining stable image quality even under drastic changes in ambient light, thereby improving biometric accuracy under various lighting conditions.

[0043] Through the embodiments provided in this application, in response to image acquisition commands, the image to be corrected and the device operating status of the smart device are obtained. The cover material properties of the smart device can be detected and identified in real time or periodically to determine the current operating status of the smart device. The cover material properties change dynamically with the physical degradation of the cover of the smart device. Furthermore, a target correction strategy matching the current operating status is selected from multiple correction strategies. The mutual influence between the device operating status and the cover material properties is fully considered to ensure that the selected target correction strategy can more accurately address the challenge of image quality degradation caused by the physical degradation of the cover. The target correction strategy is applied to the image to be corrected to perform image correction and generate a corrected image with optimized quality. This improves the input image quality of the image recognition process. The corrected image serves as the input for the image recognition function to improve the accuracy and robustness of biometric recognition. This effectively overcomes the poor image quality caused by changes in cover material properties and significantly improves the image recognition success rate of the smart device under different cover materials. Therefore, it can solve the technical problem in related technologies where the image recognition success rate decreases due to image degradation caused by cover material.

[0044] In one exemplary embodiment, the smart device includes a visible light camera, a near-infrared illumination unit, and a near-infrared camera;

[0045] In an optional embodiment, when the smart device is a smart door lock, in response to a received image acquisition command, the sensing module of the smart door lock is controlled to simultaneously acquire multispectral image data and environmental information. The sensing module includes at least some of the following components: a near-infrared illumination unit, a visible light camera, a near-infrared camera, an ambient light sensor, and a temperature sensor.

[0046] In some embodiments, the method further includes: acquiring a visible light image using a visible light camera, and simultaneously acquiring a reflected image using a near-infrared camera when a near-infrared illumination unit emits light signals of a first wavelength and a second wavelength, to obtain a first near-infrared image and a second near-infrared image; extracting an optical feature vector based on the visible light image, the first near-infrared image, and the second near-infrared image, and determining the material properties of the cover plate based on the optical feature vector.

[0047] In this embodiment, a visible light camera refers to a camera used to capture images in the visible spectrum range (approximately 380nm to 740nm). Optionally, the visible light image may include ambient color, brightness, and other characteristics within the visible light domain.

[0048] Near-infrared illumination units refer to light sources that emit near-infrared light (such as infrared LED arrays), while near-infrared cameras are cameras used to capture images in the near-infrared spectral range (approximately 740nm to 1000nm).

[0049] The near-infrared illumination unit in the smart device can emit light signals at different wavelengths (e.g., 850nm and 940nm), and the near-infrared camera can simultaneously capture the image reflected back from the cover under the illumination of these light signals to obtain the reflected image.

[0050] For example, a near-infrared illumination unit emits modulated light signals at a first wavelength (e.g., 850 nm) and a second wavelength (e.g., 940 nm), respectively, and a near-infrared camera simultaneously captures the reflected images to obtain a first near-infrared image and a second near-infrared image. The first and second near-infrared images may include the cover plate's transmittance, absorption characteristics, and scattering characteristics. Simultaneously, a visible light camera acquires a visible light image from the corresponding viewing angle; the visible light image is used for subsequent color restoration processing, while the near-infrared image is used to penetrate the cover plate and suppress ambient light interference.

[0051] In related technologies, only a single frame of near-infrared light (NIR) is collected, which is susceptible to noise interference. To solve the problems in related technologies, this embodiment acquires a first near-infrared image and a second near-infrared image at a first wavelength and a second wavelength, respectively, and improves data reliability through multi-frame fusion and synchronous detection.

[0052] Here, acquiring a first near-infrared image and a second near-infrared image at a first wavelength (e.g., 850 nm) and a second wavelength (e.g., 940 nm) respectively is to utilize the differences in transmission and reflection characteristics of near-infrared light at different wavelengths to more comprehensively evaluate the optical properties of the cover material. Different materials have different transmittance and reflectance to different wavelengths of light. For example, dark inks may have high transmittance at 850 nm, while transmittance drops significantly at 940 nm. This means that by comparing the reflection intensity of the same cover at the two wavelengths, the thickness and absorption characteristics of the ink layer can be estimated more accurately. Furthermore, near-infrared light exhibits different sensitivities to ambient light at short wavelengths (e.g., 850 nm) and long wavelengths (e.g., 940 nm). Short-wavelength near-infrared light is more susceptible to ambient light interference, while longer-wavelength near-infrared light can suppress ambient light interference to some extent. Therefore, using two wavelengths can more effectively separate the optical signal of the cover material itself, mitigating the impact of external conditions on image quality. In addition, images captured at different wavelengths can provide complementary feature information. For example, near-infrared images with a wavelength of 850 nm may be more advantageous in revealing surface texture and microstructure, while images with a wavelength of 940 nm may be more suitable for analyzing deep structures or material uniformity. By fusing information from these two types of images, richer feature vectors can be constructed, improving the accuracy and robustness of material property recognition.

[0053] After obtaining the visible light image, the first near-infrared image, and the second near-infrared image, optical feature vectors can be extracted from these images. For example, optical feature vectors may include color mean, contrast, near-infrared reflectance ratio, and texture clarity. For instance, based on the aforementioned acquisition results (i.e., the visible light image, the first near-infrared image, and the second near-infrared image), preprocessing and feature extraction are performed on the images and sensor data from each channel to obtain multiple low-dimensional feature quantities (i.e., optical feature vectors).

[0054] Optionally, visible light images can provide rich color and texture information, while near-infrared images can provide physical properties of materials, such as absorption and reflectivity. Different wavelengths of near-infrared light, due to their penetrating power and varying responses to materials, can capture unique features that help refine material classification and physical parameter estimation. By fusing multimodal information such as visible light images, first near-infrared images, and second near-infrared images, a multidimensional feature space can be constructed, enabling the material recognition model to more accurately distinguish different types of cover materials and their physical degradation states, thus facilitating precise image correction. Furthermore, complex materials (such as dark inks, gradient coatings, or metallic decorative layers) may exhibit different optical behaviors to visible and near-infrared light. For example, dark glass may appear dull and distorted under visible light, but its reflection and transmission characteristics may be more easily identified in the near-infrared band. By comparing images obtained at different wavelengths, the selective absorption and scattering effects of the material can be analyzed, which is impossible with a single-modal image. Fusing these features enhances the perception of complex material structures and is more conducive to image correction.

[0055] In this way, by simultaneously acquiring visible light and near-infrared images, identifying the material type of the cover plate (such as dark glass or metal decorative layer), and actively activating the appropriate image correction strategy (such as image correction model), high-quality face images can still be recovered even in cases of no local occlusion but image degradation, thereby improving the recognition success rate, avoiding recognition failures caused by material absorption, and improving the success rate of face recognition under low-transmittance cover plates.

[0056] Optionally, the intelligent device preprocesses the visible light image, including segmenting different objects or regions in the image and detecting points of interest (POIs) of human faces or other biometric features (such as eyes, noses, palm prints, etc.). Then, the intelligent device extracts the average brightness or reflectance intensity of the aforementioned POI regions from the visible light image, the first near-infrared image, and the second near-infrared image, respectively. By calculating the spectral ratio of the brightness of the visible light to that of the first and second near-infrared images (here, the spectral ratio reveals the material properties of the point or region by comparing the light response of the same point or region under different spectral regions), an optical feature vector containing different spectral responses is constructed. Simultaneously, the intelligent device also analyzes the texture features of the visible light image, such as sharpness and contrast, and the distribution of highlights and shadows in the near-infrared image, further enriching the dimensions of the optical feature vector. Finally, the system incorporates additional environmental variables such as ambient light intensity and temperature into the optical feature vector to comprehensively consider the impact of external conditions on imaging.

[0057] Alternatively, the visible light image, the first near-infrared image, and the second near-infrared image are used as inputs to a Convolutional Neural Network (CNN) to extract features from the Red-Green-Blue (RGB) and Near-Infrared (NIR) modes. The CNN generates feature maps for the three modes through multiple convolutional and pooling layers, which can capture local texture, shape, and structural information in the image. In the intermediate layers of the CNN, feature fusion techniques such as feature concatenation or feature weighted averaging are used to merge the feature maps of different spectra into a multimodal fusion feature map. This fusion takes into account the comprehensive performance of the cover material under different spectra. Finally, the fusion feature map is reduced to a fixed-length optical feature vector through fully connected layers or clustering algorithms.

[0058] In an optional embodiment, the acquired visible light image, first near-infrared image, and second near-infrared image are preprocessed, including image alignment, denoising, and normalization. Then, optical feature vectors are extracted from these images, including but not limited to RGB color mean, near-infrared reflectance intensity, reflectance ratio between wavelengths, image grayscale mean, and image contrast. Based on the extracted optical feature vectors, a physical model is constructed using Fresnel reflection and the Beer-Lambert law to initially estimate the physical properties of the cover material, such as refractive index, absorptivity, and scattering coefficient. A machine learning model is then constructed; this model can be a support vector machine (SVM), decision tree, random forest, or neural network, to predict the specific properties of the cover material from the optical feature vectors. During the training phase, the machine learning model uses an image dataset labeled with real material properties to learn how to map optical features to material properties. During the prediction phase, the machine learning model receives the preliminary property estimates of the cover material calculated by the physical model, along with the original optical features, as input, and outputs the corrected cover material properties. The calculation results of the physical model are combined with the prediction results of the machine learning model through weighted averaging or other fusion methods to obtain more accurate cover plate material properties.

[0059] Optionally, a material recognition model can be used to identify the material properties of the cover plate. The material recognition model can be a lightweight fully connected neural network. Its input layer receives the above 11-dimensional optical features and undergoes nonlinear transformation through at least one hidden layer. The hidden layer includes 128, 64, and 32 neurons in sequence and uses the LeakyReLU activation function. A multi-task learning head is set at the network output end to perform classification and regression tasks simultaneously.

[0060] The classification task predicts the material type of the cover plate under test, and the output categories include at least three of the following: transparent or light-colored glass, dark glass, metal, or opaque material. The probability distribution is output using the Softmax function. The regression task estimates the continuous physical properties of the cover plate, including at least one of the following: thickness, ink concentration, and infrared absorption coefficient. This is achieved using a linear output layer.

[0061] Optionally, the model training process is as follows: Collect an image dataset covering various cover materials (such as transparent / light-colored glass, dark glass, metal, or opaque materials), with at least several hundred images for each material type, ensuring the dataset covers the diversity of materials and their performance under different lighting conditions. Generate two types of labels for each image: a classification label and a regression label. The classification label reflects the cover material type; for example, transparent glass = 0, dark glass = 1, and metal = 2. The regression label describes the physical properties of the cover, such as thickness, ink density, or infrared absorption coefficient. These labels should be accurately labeled based on actual measurements or data provided by the manufacturer. Preprocess the original image data, including image alignment, correction, normalization, and extraction of optical feature vectors. Optical features may include average reflectance intensity at different wavelengths, reflectance intensity ratios, color mean of visible light images, texture sharpness indices, etc. A lightweight, fully connected neural network is constructed. The input layer receives optical feature vectors, and the hidden layers contain 128, 64, and 32 neurons respectively. The LeakyReLU activation function is used to address the vanishing gradient problem and enhance nonlinear expressiveness. The output layer uses the Softmax activation function to generate the probability distribution for material classification and employs a linear activation function to directly output the predicted physical properties. A cross-entropy loss function is used to measure the difference between the model's output probability distribution and the true classification label. Based on Fresnel's law of reflection and Beer-Lambert's law of absorption, a loss function is designed to penalize model predictions that do not conform to physical laws. For example, for transparent materials, the model should predict a lower infrared absorption coefficient; for dark materials, the model should predict a lower reflectivity. Stochastic gradient descent (SGD) or Adam optimization algorithms are used to minimize the composite loss function and iteratively update the network weights. Model performance is evaluated periodically using a validation set, monitoring metrics such as classification accuracy and regression error. Based on the validation results, the model structure (e.g., number of neurons, number of layers), optimization algorithm parameters (e.g., learning rate), and the weights of the physical constraint loss are adjusted to achieve an optimal balance. After training, the model is fully evaluated using an independent test dataset to ensure that it maintains stable and accurate predictive performance on unseen data.

[0062] In the model training phase, a composite loss function is used for optimization. This composite loss function includes classification loss and physical constraint loss. The classification loss is cross-entropy loss; the physical constraint loss is constructed based on optical physics laws to constrain the model output to conform to Fresnel's law of reflection and / or Beer-Lambert's law of absorption. Thus, the introduction of physical constraint loss ensures that the model not only relies on statistical data patterns during prediction but also follows the optical propagation characteristics of the real world, thereby improving the model's generalization ability and prediction reliability. Through this structure, this embodiment can achieve efficient and accurate material property identification on resource-constrained embedded devices, providing a reliable basis for the selection of subsequent image correction strategies and parameter generation. In related technologies, only the "transmittance" of the cover plate material property is estimated, failing to distinguish between reflection and absorption. This solution achieves decoupling through physical priors, improving estimation accuracy. It realizes effective perception of the physical information related to the cover plate material, providing reliable input for subsequent material classification and parameter estimation, while also considering low power consumption, real-time performance, and anti-interference capabilities.

[0063] In this embodiment, by comparing the reflection intensity of near-infrared images at different wavelengths, the absorption and scattering effects of the cover material can be distinguished, while visible light images are used to assess the impact of ambient light on image quality. This allows for a deeper understanding of the cover properties from a physical perspective, rather than relying solely on visual observation of surface color or texture. Furthermore, by fusing visible light and multi-wavelength near-infrared image data, efficient perception and accurate identification of the cover material properties are achieved, providing a solid foundation for dynamically selecting image correction strategies and generating correction parameters. This multimodal perception mechanism significantly improves the recognition success of biometric devices in complex environments and with diverse materials.

[0064] In one exemplary embodiment, the smart device further includes an ambient light sensor and a temperature sensor, wherein the ambient light sensor is used to collect the ambient light intensity of the smart device; and the temperature sensor is used to collect the current temperature of the smart device.

[0065] Among them, the ambient light sensor is a sensor that senses the brightness of the surrounding environment. Its working principle is to measure the light intensity in the environment, measured in lux. Optionally, the ambient light sensor can capture ambient light within the visible spectrum. For biometric systems, the ambient light sensor can monitor ambient lighting conditions in real time, providing necessary environmental information for subsequent image processing and correction. For example, the ambient light intensity is provided by the ambient light sensor. The temperature sensor measures and senses changes in the internal temperature of the smart device. The current device temperature is collected by the temperature sensor. Optionally, different temperatures may affect the performance and stability of optical components (such as infrared LEDs and cameras). For example, temperature changes affect the luminous efficiency and wavelength of infrared LEDs, thus affecting the quality of near-infrared images. Simultaneously, temperature may also affect the optical properties of the cover plate, such as refractive index and transmittance. Through real-time monitoring by the temperature sensor, the smart device can take appropriate temperature compensation measures, such as adjusting the light source intensity, activating overheat protection mechanisms, or dynamically adjusting image processing parameters.

[0066] In some embodiments, extracting an optical feature vector based on a visible light image, a first near-infrared image, and a second near-infrared image includes: determining at least one of the following optical features: the average reflectance of the first near-infrared image, the average reflectance of the second near-infrared image, the ratio of the reflectance of the first near-infrared image to that of the second near-infrared image, the signal-to-noise ratio of the first near-infrared image, the signal-to-noise ratio of the second near-infrared image, the average pixel values ​​of the red, green, and blue channels in the visible light image, the texture sharpness index of the visible light image, the proportion of specular highlight areas in the visible light image, and the image alignment quality score between the visible light image, the first near-infrared image, and the second near-infrared image; the image alignment quality score is used for the spatial registration accuracy of the visible light image, the first near-infrared image, and the second near-infrared image; and fusing at least one optical feature to obtain an optical feature vector.

[0067] In this embodiment, the average reflectance intensity of the first near-infrared image is the average value of all pixel values ​​in the first near-infrared image captured by the near-infrared camera at a specified wavelength, and the average reflectance intensity of the second near-infrared image is the average value of all pixel values ​​in the second near-infrared image captured by the near-infrared camera at another specified wavelength. Both the average reflectance intensity of the first and second near-infrared images reflect the reflectivity of the cover plate to near-infrared light. For example, the average reflectance intensity of the first near-infrared image is the average value of all pixel values ​​in the first near-infrared image captured by the near-infrared camera at a wavelength of 850 nm, and the average reflectance intensity of the second near-infrared image is the average value of all pixel values ​​in the second near-infrared image captured by the near-infrared camera at a wavelength of 940 nm.

[0068] The ratio of the reflectance intensity of the first near-infrared image to that of the second near-infrared image refers to the ratio between the average reflectance intensity of the first near-infrared image and the average reflectance intensity of the second near-infrared image.

[0069] The signal-to-noise ratio (SNR) of the first near-infrared image is the ratio of signal strength to background noise power in the first near-infrared image, and the SNR of the second near-infrared image is the same ratio. A higher SNR indicates clearer image details and higher recognition accuracy. Here, the SNR of the first and second near-infrared images are used to evaluate the image quality at different wavelengths.

[0070] The pixel mean values ​​of the red, green, and blue channels in a visible light image are the average values ​​of all pixels in each of the red, green, and blue channels. In other words, the pixel mean values ​​of the red, green, and blue channels in a visible light image include the average values ​​of all pixels in the red channel, the green channel, and the blue channel. Optionally, the pixel mean values ​​of the red, green, and blue channels in a visible light image can be used to analyze the absorption or scattering characteristics of the cover material for different colors of light, and to determine the overall illumination level and color tendency of the image.

[0071] The texture sharpness index for visible light images is a parameter used to quantify the richness and sharpness of texture details in an image, reflecting the numerical value of the clarity of surface details of objects in a visible light image. A higher texture sharpness index indicates richer texture details. For example, the texture sharpness index can be obtained by calculating the changes in local edge intensity or contrast of an image. A common method is to use image gradient operators (such as the Sobel operator, Laplacian operator, or Canny edge detection algorithm) to estimate the edge intensity of each pixel in the image, and then perform statistical analysis on these intensity values, such as calculating the mean, standard deviation, or variance, to obtain a comprehensive texture sharpness index. Another example is using the Laplacian gradient variance to measure the texture sharpness index. Assuming the Laplacian operator is used to calculate the texture sharpness index, the process is as follows: apply the Laplacian operator to each pixel of the image; calculate the Laplacian response of each pixel, i.e., the second derivative value of that pixel; square the Laplacian responses of all pixels, and then calculate the average to obtain the texture sharpness index.

[0072] The specular highlight region ratio in a visible light image is the proportion of specular highlight regions within the visible light image. A specular highlight region refers to an area within the specular highlight region whose brightness exceeds a specified threshold. Optionally, the specular highlight region ratio in a visible light image can reflect whether there is excessive reflected light on the cover surface. For example, the specular highlight region ratio can be obtained by binarizing areas exceeding a preset brightness threshold. This involves binarizing the visible light image, marking all pixels above the preset brightness threshold as 1 (representing highlight regions), and marking the remaining pixels as 0. The percentage of pixels marked as 1 in the entire image is then calculated, yielding the specular highlight region ratio.

[0073] The image alignment quality score between the visible light image, the first near-infrared image, and the second near-infrared image is a quantitative indicator used to evaluate the spatial registration accuracy of the two images. It reflects the degree of consistency of feature points or regions between the images. For example, the image alignment quality score between the visible light image, the first near-infrared image, and the second near-infrared image may include, but is not limited to, the degree of overlap between pixels, the matching degree of edge contours, and the degree of image distortion. It is understood that the image alignment quality score is calculated based on multiple frames of the first near-infrared image, the second near-infrared image, and the visible light image, comparing the positional deviation of corresponding feature regions in each frame. The smaller the deviation, the higher the image alignment quality score.

[0074] For example, the image alignment quality score between the visible light image, the first near-infrared image, and the second near-infrared image can be obtained based on feature matching algorithms, such as Scale-Invariant Feature Transform (SIFT). By detecting and describing key points in each image, and finding corresponding points between the visible light image, the first near-infrared image, and the second near-infrared image, the distance deviation, angle deviation, or similarity of feature descriptors between these corresponding points is calculated. If the deviation between corresponding points is small and the similarity of feature descriptors is high, the image alignment quality score is high; otherwise, it is low.

[0075] Spatial registration accuracy refers to the degree of precision in aligning a visible light image, a first near-infrared image, and a second near-infrared image in a spatial coordinate system. Spatial registration refers to the process of adjusting the visible light image, the first near-infrared image, and the second near-infrared image to the same reference coordinate system through geometric transformations (such as translation, rotation, and scaling) so that the feature points between the images can correspond precisely. High registration accuracy means a high image alignment quality score between the images.

[0076] For example, multiple optical features can be determined through multimodal perception, which includes acquiring three heterogeneous signals: ambient light, raw image format (RAW) images, and near-infrared reflectance. This allows for more accurate estimation of the optical properties of the cover material, thereby optimizing image correction strategies, improving recognition success rates, and jointly estimating the properties of external interference sources.

[0077] The above-mentioned multiple optical features can be fused to obtain an optical feature vector.

[0078] Optionally, the aforementioned optical features are extracted from the visible light image, the first near-infrared image, and the second near-infrared image, respectively. These features include the average reflectance intensity of the first near-infrared image, the average reflectance intensity of the second near-infrared image, the reflectance intensity ratio, the signal-to-noise ratio, the pixel mean of the red, green, and blue channels in the visible light image, the texture sharpness index, the proportion of specular highlight areas, and the image alignment quality score. The extracted features are normalized to facilitate subsequent fusion. For example, all numerical features are scaled to the same range (e.g., between 0 and 1) to eliminate the influence of dimensional differences. Furthermore, for categorical features (e.g., whether the proportion of specular highlight areas exceeds a certain threshold), they can be converted into binary codes of 0 or 1. The processed features are then concatenated together to form a one-dimensional array, which is the optical feature vector. For example, if the average reflectance of the first near-infrared image is 0.6 (after normalization), the average reflectance of the second near-infrared image is 0.8, the reflectance ratio is 0.75, the signal-to-noise ratios are 0.9 and 0.85 (after normalization), the pixel mean values ​​are 0.2, 0.3, and 0.4, the texture clarity index is 0.8, the specular highlight area ratio is 0.1 (converted to 0), and the image alignment quality score is 0.95, then the generated optical feature vector may be represented as [0.6, 0.8, 0.75, 0.9, 0.85, 0.2, 0.3, 0.4, 0.8, 0, 0.95].

[0079] In an optional embodiment, multiple optical features are fused to generate a multi-dimensional feature vector (i.e., an optical feature vector). The optical features include at least: the average reflectance of a first near-infrared image in a first near-infrared band (e.g., 850 nm), the average reflectance of a second near-infrared image in a second near-infrared band (e.g., 940 nm), the ratio of the first to the second near-infrared image, the average color value of the visible light image, the texture sharpness index, the ambient light intensity, and the signal-to-noise ratio of the first and second near-infrared images. All of the above optical features, after normalization, constitute a one-dimensional 11-dimensional optical feature vector, which can be used as input to the aforementioned material recognition model.

[0080] Furthermore, for example, during image acquisition, a frame synchronization mechanism can be used to ensure precise alignment between multi-wavelength illumination and image capture; and motion artifacts can be compensated to improve the stability of feature extraction, thus solving the problem of global optical interference (fixed or semi-fixed material properties).

[0081] In this way, by fusing near-infrared and visible light multimodal features to construct an 11-dimensional physical perception feature vector, which serves as the input to the material recognition model, and by introducing optical physical laws as regularization constraints during training, the model achieves joint and reliable recognition of the material (transparent / dark / metallic) and physical parameters (such as thickness and ink concentration) of the door lock cover, thereby improving the physical consistency and generalization ability of subsequent image correction. Furthermore, by simultaneously acquiring images in the first near-infrared band (e.g., 850nm), the second near-infrared band (e.g., 940nm), and visible light, the model calculates the ratio of their reflectance intensity and texture clarity index, which serve as key input features for material discrimination. This enhances the ability to distinguish between dark glass and metallic materials, and allows for material classification using multispectral information, resulting in a stronger physical understanding capability.

[0082] In this embodiment, by integrating data from ambient light and temperature sensors, the intelligent device can comprehensively perceive the state of the imaging environment and the health status of the device. By analyzing the optical features of visible light images, first near-infrared images, and second near-infrared images, a multi-dimensional optical feature vector is constructed. This not only captures real-time image quality and material properties but also integrates ambient light intensity and device temperature changes, providing a solid foundation for subsequent dynamic adjustment of image correction strategies and model selection. Under various physical degradation and environmental fluctuations, it can accurately identify and intelligently compensate for image distortion, optimizing the accuracy and response speed of biometric recognition.

[0083] In an exemplary embodiment, the device operating state includes the current battery level and system operating mode of the smart device; determining a target correction strategy that matches the current operating state from multiple correction strategies includes: in response to the current operating state of the smart device being a power-saving operating state, determining a first correction strategy from multiple correction strategies as the target correction strategy that matches the power-saving operating state; the power-saving operating state refers to the operating state where the current battery level of the smart device is lower than a preset power threshold, or the operating state where the system operating mode is in a low-power operating mode; the first correction strategy refers to a strategy for image correction based on a target lookup table; the target lookup table records the pixel value mapping relationship between the image to be corrected and the corrected image obtained by the smart device under the cover material properties.

[0084] In this embodiment, the current battery level refers to the remaining power of the battery built into the smart device. For example, the current battery level is displayed as a percentage. The system operating mode is the operating state that the smart device changes according to current needs, usage environment, or preset strategies. Optionally, the system operating mode includes normal mode, low-power operating mode, performance mode, night mode, etc.

[0085] If the current battery level of the smart device is lower than a preset power threshold, the current operating condition of the smart device is determined to be a power-saving condition; or, if the smart device's system operating mode is in a low-power operating mode, the current operating condition of the smart device is determined to be a power-saving condition. After determining that the current operating condition of the smart device is a power-saving condition, the first correction strategy among multiple correction strategies is selected as the target correction strategy to match the power-saving condition. The first correction strategy refers to an image correction strategy based on a target lookup table, that is, the first correction strategy can be a three-dimensional lookup table (3D LUT) image correction method; wherein, the target lookup table records the pixel value mapping relationship between the image to be corrected and the corrected image obtained by the smart device under the current cover material properties.

[0086] Thus, under power-saving conditions, smart devices need to make efficient use of limited energy. Therefore, target lookup tables are chosen as the image correction strategy under this power-saving condition. By using pre-stored mapping relationships to correct colors, during image correction, the smart device only needs to look up the output value of the corresponding input pixel in the table, without the need for complex mathematical operations or forward propagation of deep learning models. This greatly reduces computational power consumption, extends the device's battery life in low-power or low-battery states, and avoids additional power consumption caused by computational resource consumption.

[0087] Optionally, the pixel value mapping relationship between the image to be corrected and the corrected image in the target lookup table can be set before the smart device leaves the factory or updated based on historical data. However, using factory-preset white balance parameters or a fixed LUT (lookup table) for color compensation cannot dynamically adapt to changes in the device's environment, and the fixed color compensation method cannot cope with global image degradation caused by materials, which may have the following obvious defects: it cannot adapt to changes such as user-applied films, oil stains, aging and discoloration; the difference in ink thickness between different batches leads to inconsistent light transmission characteristics; fixed parameters are difficult to cope with complex materials such as gradient coatings and metallic paints. To solve the above problems, the embodiments of this application dynamically adjust the smart device to a first correction strategy based on the target lookup table under node conditions, and the mapping relationship in the target lookup table changes dynamically according to the cover material properties of the smart device. This means that the mapping relationship in the lookup table is not fixed, but can be dynamically adjusted according to the actual imaging effect of the device under different materials and different ambient light conditions, which can effectively correct the image color of the smart device.

[0088] In an optional embodiment, if the current battery level is below a preset threshold (e.g., 20%), or the system is in a low-power operating mode, the image correction method based on a target lookup table (i.e., 3D LUT) is preferentially enabled. Specifically, the process of correcting the image to be corrected according to the target lookup table is as follows: A series of pre-built lookup table templates are stored internally or in the cloud. These lookup table templates cover image correction mappings for different material properties and operating conditions, such as transparent materials, dark materials, and metallic materials. Based on the estimated cover material properties (e.g., transmittance, ink concentration, infrared absorption coefficient), the system dynamically selects the lookup table that best matches the current operating condition as the target lookup table. For example, if the current cover transmittance is 30%, the lookup table in the pre-stored LUT that is for a transmittance of around 30% is selected. In low-power mode, the parameters of the LUT may be further adjusted to reduce computation and memory usage, for example, by reducing the resolution of the lookup table or using a simplified version of the lookup table. Each pixel value of the image to be corrected (usually in RGB color space) is used as input, and the corresponding output value is searched in the target lookup table. In this process, a Lookup Table (LUT) is typically a three-dimensional matrix, corresponding to the pixel values ​​of the RGB color channels. For each pixel in the input image, a fast lookup in the LUT finds the mapping output that best matches the input pixel value, achieving color space conversion. For each pixel in the input image, a lookup table is used to map the corrected color value, which replaces the original pixel value, thus completing the image correction for the entire image. After these pixel-level image correction operations, the generated image is the corrected image to be corrected, whose color quality is closer to the effect under real-world lighting conditions, reducing color deviation and contrast loss caused by material properties.

[0089] Through this embodiment, by dynamically adjusting to the first correction strategy based on the target lookup table, the smart device can achieve high efficiency and energy saving of image correction function under power-saving conditions. This not only extends the effective working time of the smart device in low power conditions, but also ensures the quality of image processing without requiring a large amount of computing resources, thereby improving the device's intelligent response capability.

[0090] In an exemplary embodiment, the method further includes: selecting a first lookup table and a second lookup table from a lookup table template library that match the cover plate material attribute; associating the first lookup table with the first material attribute; associating the second lookup table with the second material attribute; determining a first absolute value of the difference between the second material attribute and the cover plate material attribute, and determining a second absolute value of the difference between the first material attribute and the second material attribute; determining the ratio between the first absolute value of the difference and the second absolute value of the difference as a first interpolation ratio of the first lookup table; performing a complement operation on the first interpolation ratio to obtain a second interpolation ratio of the second lookup table; the sum of the first interpolation ratio and the second interpolation ratio is 1; creating an empty table, determining each color channel in the empty table as the current color channel, and performing the following interpolation operations to obtain a target lookup table: extracting the first pixel value corresponding to the color channel that is the same as the current color channel from the first lookup table, and extracting the second pixel value corresponding to the color channel that is the same as the current color channel from the second lookup table; determining a first product between the first interpolation ratio and the first pixel value, and determining a second product between the second interpolation ratio and the second pixel value; and determining the sum of the first product and the second product as the target pixel value corresponding to the current color channel.

[0091] In this embodiment, the lookup table template library stores multiple lookup tables, each matching a specific cover material attribute; that is, each lookup table matches a different cover material attribute. Here, because the cover material attribute dynamically changes with the physical degradation of the cover during operation, linear interpolation can dynamically adjust the mapping relationship in the lookup table based on the specific material attribute of the current cover, improving the accuracy of the correction effect. Furthermore, the pre-built lookup table template library may not completely cover all possible material attribute changes. Linear interpolation can effectively utilize the neighboring attribute values ​​of existing lookup tables to generate a target lookup table for the specific characteristics of the current cover, compensating for the insufficient coverage of the LUT library. Therefore, linear interpolation is performed on the first and second lookup tables respectively.

[0092] Select a first lookup table and a second lookup table from the lookup table template library. The first lookup table is associated with a first material property, and the second lookup table is associated with a second material property. For example, the first lookup table is associated with material properties with low light transmittance or strong absorption, while the second lookup table is associated with material properties with high light transmittance or weak absorption.

[0093] The first interpolation ratio of the first lookup table is the ratio between the absolute values ​​of the first and second differences. The second interpolation ratio of the second lookup table is the ratio obtained by complementing the first interpolation ratio. Optionally, the second interpolation ratio is obtained by subtracting 1 from the first interpolation ratio. That is, the sum of the first and second interpolation ratios is 1.

[0094] A color channel refers to a single element that makes up the color information of a digital image. Optionally, in an image format, color is composed of three color components, each corresponding to a different color channel. For example, the Red, Green, and Blue (RGB) color model uses three color channels to represent the color information in an image. The Red Channel represents the intensity of the red component in the image; the Green Channel represents the intensity of the green component; and the Blue Channel represents the intensity of the blue component. In other words, a color channel can include a red channel, a green channel, and a blue channel.

[0095] In an optional embodiment, when using a 3D LUT-based image correction method, two lookup tables (i.e., a first lookup table and a second lookup table) that best match the current material properties are selected from a LUT template library pre-stored in an off-chip storage unit. For example, the first lookup table and the second lookup table correspond to cover material with transmittance of 30% and 50%, respectively. Based on the estimated equivalent transmittance or absorption coefficient, linear interpolation is performed on the first lookup table and the second lookup table to generate a target LUT adapted to the current scene. The target LUT has a size of 33×33×33 and maps the input RGB color space to the output corrected RGB values. A color lookup operation is performed to complete the pixel-level color mapping of the image.

[0096] Optionally, the system's lookup table template library stores multiple standard LUTs matched with different transmittances or absorption coefficients. For example, a first lookup table LUT(_1) with a transmittance of 30% and a second lookup table LUT(_2) with a transmittance of 50%. Based on the currently estimated transmittance τ (i.e., the cover material property), the first interpolation ratio ω(_1) and the second interpolation ratio ω(_2) of LUT(_1) and LUT(_2) are calculated, where ω(_1) + ω(_2) = 1. The interpolation ratios are determined by a linear relationship: ω(_1) = (τ(_2) - τ) / (τ(_2) - τ(_1)), ω(_2) = 1 - ω(_1). For example, suppose the transmittance (i.e., the first material property) of LUT1 is τ1 = 30%, the transmittance (i.e., the second material property) of LUT2 is τ2 = 50%, and the currently estimated transmittance (i.e., the cover material property) is τ = 40%. The first interpolation ratio ω1 and the second interpolation ratio ω2 are calculated as follows: ω1=(τ2-τ) / (τ2-τ1)=(50%-40%) / (50%-30%)=10% / 20%=0.5, ω2=1-ω1=1-0.5=0.5. For each RGB color channel, the corresponding pixel mapping relationship is selected from the first lookup table LUT(_1) and the second lookup table LUT(_2). According to the calculated first interpolation ratio ω1 and the second interpolation ratio ω2, linear interpolation is performed point-by-point within the input color space. For a certain input value x (i.e., the pixel value corresponding to the color channel with the same color channel as the current color channel), the corresponding output value y(_1)(x) (i.e., the first pixel value) is found from the first lookup table LUT(_1); the corresponding output value y(_2)(x) (i.e., the second pixel value) is found from the second lookup table LUT(_2). The interpolated output value y(x) (i.e., the target pixel value) = ω(_1) is calculated. y(_1)(x)+ω(_2) y(_2)(x), for example, taking the input RGB pixel value (25,50,75) as an example, assuming that the outputs of LUT1 and LUT2 corresponding to this pixel value are (35,65,95) and (45,75,105) respectively, the interpolated output value is calculated as follows: Interpolated red channel target pixel value yR=ω1 y1R+ω2 y²R = 0.5 35+0.5 45=40; Interpolate the target pixel value of the green channel yG=ω1 y1G+ω2 y²G = 0.5 65+0.5 75=70; Interpolate the target pixel value of the blue channel yB=ω1 y1B+ω2 y²B = 0.5 95 + 0.5 Since 105 = 100, the interpolated target LUT corresponds to an output value of (40, 70, 100) for an input of (25, 50, 75). Repeat the above operation, performing linear interpolation on each pair of input and output values ​​in LUT1 and LUT2, until a complete target LUT is generated, with the same dimensions as LUT1 and LUT2, for example, 33×33×33.

[0097] In this way, by dynamically selecting and generating an appropriate image correction model based on the identified cover material type and physical parameters, a fast correction path based on 3D LUT interpolation is enabled in low-power mode.

[0098] In this embodiment, by implementing dynamic lookup table interpolation based on the material properties of the cover plate, the system can flexibly adapt to image degradation caused by different cover plate materials. Compared with using only a single lookup table, this embodiment can more accurately adjust the image correction strategy, ensuring that the corrected image quality is neither over-corrected nor under-corrected, providing a smoother and more accurate image correction effect, and improving the controllability of image quality.

[0099] In an exemplary embodiment, the device operating state includes the number of image recognition failures of the smart device; determining a target correction strategy that matches the current operating state from multiple correction strategies includes: responding to the current operating state of the smart device being a complex operating state, and determining a second correction strategy from multiple correction strategies as the target correction strategy that matches the complex operating state; a complex operating state refers to an operating state where the number of image recognition failures is greater than a preset threshold, or an operating state where the material properties of the cover plate are greater than a preset property threshold; the second correction strategy refers to a strategy for image correction of the image to be corrected using an image correction model; the image correction model is used to perform color compensation on the image to be corrected and output the corrected image.

[0100] In this embodiment, the number of image recognition failures of the smart device refers to the number of times the smart device fails to successfully recognize the target (such as a face, palm print, or other biometric features) during image recognition operations. Optionally, the image recognition failure of the smart device may be caused by a variety of factors, including but not limited to insufficient image clarity and unclear features; poor ambient lighting conditions affecting image quality; occlusion or angle changes of the target object (such as a face); and absorption or scattering of light by the cover material (such as dark glass or metal decorative frame), leading to image degradation.

[0101] If the number of image recognition failures exceeds a preset threshold, or if the cover material properties exceed a preset attribute threshold, the current operating condition of the smart device is determined to be a complex operating condition. A second correction strategy is selected from multiple correction strategies and designated as the target correction strategy for the complex operating condition. The second correction strategy refers to the image correction strategy performed on the image to be corrected using an image correction model. This image correction model is based on a deep learning algorithm that learns and understands the color variation patterns of an image to predict and adjust its color representation to achieve the desired output standard. It is used to perform color compensation on the image to be corrected and output the corrected image.

[0102] Here, the intelligent device is operating under complex conditions, indicating that interference with the device is relatively severe. To improve the accuracy of image recognition, an image correction model is used to correct the image. The image correction model has the ability to learn online and optimize adaptively. It can dynamically adjust the correction strategy based on the current number of image recognition failures, the material properties of the cover plate, and the operating status. It can adapt to the current image degradation by fine-tuning its own parameters, thereby improving the recognition success rate. In scenarios with high recognition failure rates or abnormal cover plate material properties, the image correction model can provide depth image restoration. For example, depth image restoration may include, but is not limited to, image correction, brightness equalization, contrast enhancement, and texture clarity improvement.

[0103] Optionally, if the number of consecutive failures of face recognition reaches two or more, or if the estimated medium absorption coefficient (i.e., cover material property) is greater than a preset absorption threshold (e.g., 0.8), then a refined color compensation method based on a neural network (image correction model) is enabled.

[0104] For example, when employing a refined color compensation method based on neural networks, a lightweight neural network model, denoted as LockColorNet, is used. Its network structure includes: an input layer that receives a single-channel near-infrared aligned image with dimensions 1920×1080×1; a modulation convolutional layer whose kernel weights are dynamically generated by a small MLP based on current image features; multiple residual connection modules to mitigate the vanishing gradient problem; a pixel shuffle upsampling module to restore the original resolution; and an output layer that outputs an RGB-corrected image with dimensions 1920×1080×3. LockColorNet can be quantized using INT8 and deployed on the device's NPU (Neural Processing Unit) to reduce computational power consumption and improve inference efficiency.

[0105] Thus, through the above embodiments, a correction strategy is matched for each operating condition: a first correction strategy is matched for energy-saving conditions, and a second correction strategy is matched for complex operating conditions, constructing a dual-path correction architecture (LUT fast path + lightweight neural network high-precision path) to achieve adaptive optimization. Simultaneously, a refined compensation path based on a lightweight neural network is enabled in high-interference scenarios to achieve an adaptive balance between resources and accuracy; and the introduction of the aforementioned dedicated image restoration model (such as the image correction model) can effectively address severe color distortion caused by material absorption / scattering.

[0106] Through this embodiment, by introducing a monitoring mechanism for the number of image recognition failures and an intelligent recognition and response mechanism for complex working conditions, the intelligent device can adjust its image correction strategy in a timely manner, improving recognition performance under difficult conditions. Under complex working conditions, the intelligent device activates a second correction strategy, using an image correction model to handle image color deviations. Through the powerful nonlinear mapping capability of the deep learning model, high-quality image restoration is achieved, thereby improving the accuracy and stability of image recognition. This enables the device to make more refined and precise adjustments when facing highly difficult image correction tasks, avoiding recognition failures caused by improper image processing.

[0107] In an exemplary embodiment, when a smart device is used for the first time, the image correction model may lack sufficient local training data or correction history, which may cause the initial state of the model to be unable to adapt to the current working environment immediately. To solve the above problem, a zero-sample generation mechanism is adopted, and a conditional encoder is introduced to predict the initial parameters of the image correction model based on the optical features of the current image. The smart device can achieve rapid model startup and configuration, avoiding long waiting times for model training, and enabling the smart device to perform high-quality image correction.

[0108] In some embodiments, the device operating status also includes a version identifier of the image correction model.

[0109] In this embodiment, the version identifier of the image correction model is used to mark whether the model is in its original factory state. Optionally, when the image correction model is in the initial identifier state, it indicates that the smart device is using it for the first time or the model parameters have not been updated.

[0110] In some embodiments, before performing image correction on the image to be corrected using the target correction strategy, the method further includes: in response to the version identifier of the image correction model being an initial identifier, performing feature extraction on the image to be corrected to obtain an optical feature vector, inputting the optical feature vector into a pre-trained conditional encoder to obtain initial model parameters of the image correction model, and configuring the model parameters of the image correction model according to the initial model parameters; in response to the version identifier of the image correction model not being an initial identifier, or the initial model parameters of the image correction model being configured, continuing to perform the step of performing image correction on the image to be corrected using the target correction strategy.

[0111] In this embodiment, the conditional encoder is a machine learning model that predicts specific model parameter configurations based on feature vectors. Optionally, the conditional encoder is used to predict the initial model parameters of the image correction model.

[0112] Optionally, after obtaining the initial model parameters of the image correction model, the system automatically configures the model parameters of the image correction model according to the initial model parameters output by the conditional encoder, so that the image correction model performs color compensation on the image to be corrected according to the model parameters and outputs the corrected image to be corrected.

[0113] For example, if the current image correction model is in its initial state (i.e., the smart device is using it for the first time or the model has not yet undergone any fine-tuning), a zero-shot parameter generation mechanism is used to achieve rapid startup without historical training data. This zero-shot parameter generation mechanism generates correction model parameters directly from single-frame statistical features using a pre-trained conditional encoder, eliminating the need for historical data. When using this mechanism, a pre-trained conditional encoder, denoted as E_cond, receives the statistical feature vector of the current image (such as the aforementioned 11-dimensional optical features). This conditional encoder is a small multilayer perceptron (MLP), and its output is the initial model parameters of LockColorNet, including at least one of the following: convolutional layer offset, channel gain factor, and activation function scaling factor. These initial model parameters are not obtained through backpropagation training but are directly predicted by the conditional encoder, thus achieving rapid initialization without historical data. Furthermore, the conditional encoder is pre-learned during the training phase by simulating various material and lighting conditions and remains frozen during the inference phase, not participating in parameter updates. Through the above dynamic selection mechanism, this embodiment can intelligently switch the correction strategy according to the context state of the smart device, and achieve the best color reproduction effect in different scenarios such as low power consumption, high precision, and cold start, while taking into account system efficiency, response speed and image fidelity.

[0114] In this way, by deploying a zero-shot parameter generation mechanism on the device side, the conditional encoder directly predicts the initial calibration model parameters based on the current image features, without relying on historical training data, thus achieving rapid initialization and availability assurance for new devices or in cold start states. Furthermore, by introducing a zero-shot parameter generation mechanism to directly predict initial calibration parameters based on input image features, new devices possess effective image recovery capabilities from the first use, eliminating the need to wait for failure feedback and achieving rapid response during cold starts.

[0115] If the version identifier of the image correction model is not the initial identifier, or if the initial model parameter configuration of the image correction model is completed, continue to execute the above steps of using the target correction strategy to correct the image to be corrected. This can be understood with reference to the above embodiments, and will not be repeated here.

[0116] In this embodiment, by introducing a conditional encoder to predict the initial parameters of the image correction model based on the optical features of the current image, the smart device can achieve rapid model startup and configuration, especially when the device is used for the first time or when the model parameters are in the initial state. This avoids the long waiting time for model training and enables the device to perform high-quality image correction.

[0117] In some optional embodiments, the aforementioned multimodal biometric module switching technologies lack a model adaptive mechanism, making it impossible to optimize performance over time. This is because these technologies employ only a fixed recognition process without any feedback learning or parameter adjustment mechanisms. Even with multiple recognition failures, the system does not automatically optimize image processing strategies. It cannot adapt to long-term changes such as equipment aging (e.g., infrared lamp attenuation) and environmental changes (e.g., temperature and humidity affecting light transmittance), leading to gradual performance degradation and increased maintenance costs over time. A reliable feedback-driven online fine-tuning method is implemented. In response to the completion of parameter generation and application of the image correction model, image rendering is performed to generate the target corrected image. The reliable feedback determines whether model fine-tuning is triggered based on the consistency between recognition confidence and ambient light, preventing misleading by attacked samples.

[0118] For example, the selected and configured image correction model is applied to the original acquired visible light image or near-infrared aligned image to generate an RGB format image equivalent to the real ambient lighting conditions, denoted as the corrected image. The corrected image outperforms the uncorrected image in terms of color consistency, contrast, and detail clarity, especially significantly improving facial texture recognizability in scenes with dark colors or metallic cover plates. Further, post-processing operations are performed on the corrected image, including: white balance adjustment to match the standard D65 light source; local contrast enhancement using the CLAHE (Contrast Limiting Adaptive Histogram Equalization) method; automatic facial region focusing and super-resolution reconstruction, applied only to detected facial regions to improve the resolution of key features; and conversion of the output image format to YUV420 or RGB565 to adapt to the input requirements of downstream face recognition engines. The generated corrected image is then input into the face recognition module for authentication processing. The face recognition module includes: a face detection submodule for locating face regions in an image; a feature extraction submodule for extracting face embedding vectors using a lightweight convolutional neural network (such as MobileFaceNet); a matching and comparison submodule for calculating the similarity between the extracted embedding vectors and locally stored registration templates, using cosine distance or Euclidean distance as the metric; and a verification decision submodule for determining whether the similarity is greater than a preset threshold. If so, it outputs a verification pass signal and triggers a door lock opening command.

[0119] Optionally, in a complete recognition process, if face recognition fails multiple times consecutively (e.g., three times), and the current cover plate has been identified as a high-absorption material (such as dark glass or metal), an enhancement mode is triggered: the process described in the above embodiment is re-executed, the illumination intensity and correction strategy are dynamically adjusted, and a new corrected image is generated for the next round of recognition, until success is achieved or the maximum number of retries is reached. Furthermore, the system is also configured with a feedback mechanism: after each successful face recognition, the corrected image, original image, material recognition result, model parameters, and recognition score are packaged into a training sample and cached in local non-volatile memory for subsequent incremental learning or cloud model updates. Through the above methods, this embodiment achieves end-to-end image quality optimization and identity authentication closed loop, improving the recognition success rate and user experience under complex material interference; and by using the corrected image for face recognition, and triggering the enhancement mode when consecutive recognition failures occur and a high-absorption material is detected, re-executing the material recognition and correction process to generate an optimized image, a "perception-correction-recognition-feedback" closed loop is formed, enabling proactive responses to complex interference scenarios.

[0120] In some optional embodiments, this embodiment also provides an online model fine-tuning method based on feedback data, used to incrementally optimize the local model using verification results in actual use scenarios during the operation of intelligent access control devices, thereby achieving continuous performance improvement. In response to completing a full face recognition process (whether successful or not), the system determines whether the model update triggering conditions are met; if so, a local fine-tuning mechanism is initiated. Specifically, when any of the following conditions are met, a model update sample is generated and cached: face recognition is successful, and the current image quality score is higher than a preset threshold (e.g., signal-to-noise ratio > 25dB, texture clarity > 50); the number of recognition failures reaches a preset retry limit (e.g., three times), and the ambient light intensity is within a normal range (e.g., 100–1000 lux), excluding extreme low-light interference; the user manually triggers a "re-register" operation, indicating that the current model output does not match expectations. The model update sample includes at least: the original visible light image, infrared image, the recognition result of the current material properties, image comparison before and after image correction, system status information (e.g., battery level, temperature), and corresponding correction model parameters. The local fine-tuning process is initiated when one of the following conditions is met: the number of update samples accumulated in the cache reaches a preset sample threshold; the device is idle and connected to a stable power source (e.g., charging); the current ambient temperature is within a safe training range (e.g., 15°C–40°C) to prevent overheating and frequency throttling; and the system load is below a preset threshold to ensure that real-time functionality is not affected. In this way, by supporting online fine-tuning on the terminal, adapting to device aging, and utilizing successfully recognized samples, the final layer of the image correction model is fine-tuned when the device is idle.

[0121] The local fine-tuning process includes: updating the parameters of some network layers of a lightweight neural network (such as LockColorNet); fine-tuning only the last K layers of the network (e.g., K=2~4) while keeping the remaining layers frozen to reduce computational overhead; performing single or multiple rounds of gradient descent optimization using a small learning rate (e.g., 1e-5 to 1e-4); and using loss functions, including pixel-level reconstruction loss (e.g., L1 Loss) and perceptual loss, to measure the difference between the corrected image and the target image.

[0122] Furthermore, to prevent catastrophic forgetting, the system retains a small number of historical samples (e.g., 2-3 samples per material type) as a replay buffer. The old and new data are mixed for training during each fine-tuning. This enables the image correction model to have continuous learning and adaptive capabilities to adapt to device aging and environmental changes, supporting performance optimization during long-term use.

[0123] After fine-tuning, evaluate the performance of the updated model: if the metrics improve on the validation set (e.g., PSNR increases by ≥0.5dB, recognition accuracy increases by ≥1%), replace the original model; otherwise, roll back to the original version and record the reason for the update failure.

[0124] Optionally, anonymized model update samples and training logs are periodically uploaded to a remote server for global model aggregation and federated learning updates; the new version of the model returned by the server is automatically downloaded and prepared for activation the next time the device is idle.

[0125] Through the above mechanism, this embodiment achieves secure, efficient, and continuous optimization of the image correction and recognition model on the device side, adapting to environmental changes and hardware aging during long-term user use, and significantly improving the system's long-term robustness and personalization capabilities. Furthermore, by using successfully recognized samples to fine-tune the local correction model online when the device is idle and safety conditions are met, updating only the last few layers of the network, and combining this with a playback buffer to prevent catastrophic forgetting, the model can continuously evolve without affecting real-time functionality. Simultaneously, a terminal-side online fine-tuning mechanism is designed to update model parameters based on successfully recognized samples, preventing performance degradation.

[0126] Optionally, Figure 3 This is a flowchart of an optional image correction method according to an embodiment of this application, such as... Figure 3 As shown, the process of the above embodiment can be as follows: multimodal data acquisition (ambient light + RAW + NIR), joint estimation of material properties (including physical priors), dynamic selection or generation of calibration models (LUT / CNN / Zero-Shot), output of calibration images → face recognition → credibility feedback judgment; determine whether fine-tuning is needed. If no fine-tuning is needed, the process ends. If fine-tuning is needed, LoRA fine-tuning model (local update) is used for fine-tuning, followed by model persistence, and then the process ends. Furthermore, Figure 4 This is an optional system structure block diagram according to an embodiment of this application, such as... Figure 4As shown, this embodiment also provides a smart lock image color dynamic correction system based on multimodal perception. The system structure may include a multimodal acquisition module 401, a material property estimation module 402, a dynamic correction model generation module 403, a reliable feedback and fine-tuning module 404, and a system-level collaborative optimization module 405. Each module communicates through shared memory and an interrupt mechanism and runs on the smart lock main control SoC. The multimodal acquisition module 401 is used to acquire visible light images through a visible light camera and simultaneously acquire reflected images through a near-infrared camera when the near-infrared illumination unit emits light signals of the first and second wavelengths, thus obtaining the first and second near-infrared images. The material property estimation module 402 is used to extract optical feature vectors based on the visible light images, the first and second near-infrared images, and determine the material properties of the cover plate based on the optical feature vectors. The dynamic correction model generation module 403 is used to determine the target correction strategy that matches the current working condition from a variety of correction strategies based on the current working condition of the smart device. The reliable feedback and fine-tuning module 404 is used to determine whether to trigger model fine-tuning based on the recognition confidence and the consistency with the ambient light, to prevent being misled by attacked samples. The system-level collaborative optimization module 405 is used to integrate and optimize the operating efficiency and collaborative effect of each module in the system, ensuring that the entire system operates stably and efficiently under various complex working conditions.

[0127] The image correction method for a smart device in this application embodiment will be explained below with reference to an optional example. In this optional example, the smart device is a smart lock, and the cover material is dark gray ink (light transmittance ≈ 30%).

[0128] In this optional example, the image correction of the device is a scheme that senses the medium characteristics in real time and dynamically corrects the image color. It switches the correction path according to the material type—using 3D LUT interpolation (low power consumption) for mild interference and lightweight neural network (high precision) for severe interference, dynamically selecting the correction path to balance accuracy and power consumption. Material perception and image restoration are completed before recognition, forming a feedforward process of "perception → correction → recognition". It has the ability to compensate in advance, prevent recognition failure, reduce false positives and retries due to poor image quality, significantly improve the first recognition pass rate, and the model can adapt to long-term changes such as infrared lamp attenuation and slight pollution, maintaining long-term performance stability.

[0129] Optionally, the image correction method for smart devices may include the following steps:

[0130] Step 1: Acquire ambient light (Re=100, Ge=95, Be=90), RAW mean (μR=12, μG=10, μB=8), R NIR =0.45;

[0131] Step 2: Model output c^=dark gray, τ^(550)=0.32, α^=0.75;

[0132] Step 3: Since this is the first time using the image correction model, the current version is marked as initial. Zero-sample generation is enabled, and the initial corrected image is output.

[0133] Step 4: Identify confidence s=0.55, no fine-tuning triggered; For subsequent frames, continuous acquisition is possible. If the number of failures is ≥2, LoRA fine-tuning is triggered, thereby converging the model and improving the model recognition rate to above 0.9. LoRA fine-tuning is a lightweight parameter update method that only trains low-rank matrices and is suitable for online learning on edge devices.

[0134] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0136] According to another aspect of the embodiments of this application, an image correction apparatus for a smart device is also provided. This image correction apparatus for a smart device can be used to implement the image correction method for a smart device provided in the above embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0137] Figure 5 This is a structural block diagram of an optional image correction device for a smart device according to an embodiment of this application, such as... Figure 5 As shown, the image correction device of the smart device includes an acquisition unit 502, a determination unit 504, and an execution unit 506.

[0138] The acquisition unit 502 is used to acquire the image to be corrected from the smart device and the device operating status of the smart device in response to the image acquisition command.

[0139] The determining unit 504 is used to determine the current working status of the smart device based on the device's operating status and the material properties of the cover plate; the material properties of the cover plate are the material properties of the cover plate of the smart device; the material properties of the cover plate are obtained by detection in response to the image acquisition command or by periodic detection according to the preset acquisition cycle; the material properties of the cover plate change dynamically with the physical degradation of the cover plate.

[0140] The execution unit 506 is used to determine the target correction strategy that matches the current working condition from a variety of correction strategies, and to use the target correction strategy to perform image correction on the image to be corrected to obtain the corrected image, wherein the corrected image is used for image recognition.

[0141] It should be noted that the acquisition unit 502 in this embodiment can be used to execute the above step S202, the determination unit 504 in this embodiment can be used to execute the above step S204, and the execution unit 506 in this embodiment can be used to execute the above step S206.

[0142] Through the embodiments provided in this application, in response to image acquisition commands, the image to be corrected and the device operating status of the smart device are obtained. The cover material properties of the smart device can be detected and identified in real time or periodically to determine the current operating status of the smart device. The cover material properties change dynamically with the physical degradation of the cover of the smart device. Furthermore, a target correction strategy matching the current operating status is selected from multiple correction strategies. The mutual influence between the device operating status and the cover material properties is fully considered to ensure that the selected target correction strategy can more accurately address the challenge of image quality degradation caused by the physical degradation of the cover. The target correction strategy is applied to the image to be corrected to perform image correction and generate a corrected image with optimized quality. This improves the input image quality of the image recognition process. The corrected image serves as the input for the image recognition function to improve the accuracy and robustness of biometric recognition. This effectively overcomes the poor image quality caused by changes in cover material properties and significantly improves the image recognition success rate of the smart device under different cover materials. Therefore, it can solve the technical problem in related technologies where the image recognition success rate decreases due to image degradation caused by cover material.

[0143] In one exemplary embodiment, the smart device includes a visible light camera, a near-infrared illumination unit, and a near-infrared camera; the device includes a first extraction unit, configured to acquire a visible light image through the visible light camera, and simultaneously acquire a reflected image through the near-infrared camera when the near-infrared illumination unit emits light signals of a first wavelength and a second wavelength, thereby obtaining a first near-infrared image and a second near-infrared image; and extract an optical feature vector based on the visible light image, the first near-infrared image, and the second near-infrared image, and determine the material properties of the cover plate based on the optical feature vector.

[0144] In one exemplary embodiment, the smart device further includes an ambient light sensor and a temperature sensor. The ambient light sensor is used to collect the ambient light intensity of the smart device; the temperature sensor is used to collect the current temperature of the smart device; and a first extraction unit is used to determine at least one of the following optical features: the average reflection intensity of a first near-infrared image, the average reflection intensity of a second near-infrared image, the ratio of the reflection intensity of the first near-infrared image to that of the second near-infrared image, the signal-to-noise ratio of the first near-infrared image, the signal-to-noise ratio of the second near-infrared image, the average pixel values ​​of the red, green, and blue channels in the visible light image, the texture clarity index of the visible light image, the proportion of specular highlight area in the visible light image, and the image alignment quality score between the visible light image, the first near-infrared image, and the second near-infrared image; the image alignment quality score is used for the spatial registration accuracy of the visible light image, the first near-infrared image, and the second near-infrared image; and the at least one optical feature is fused to obtain an optical feature vector.

[0145] In an exemplary embodiment, the device operating state includes the current battery level and system operating mode of the smart device; the execution unit is further configured to, in response to the current operating state of the smart device being a power-saving operating state, determine a first correction strategy among multiple correction strategies as a target correction strategy matching the power-saving operating state; the power-saving operating state refers to the operating state where the current battery level of the smart device is lower than a preset power threshold, or the operating state where the system operating mode is in a low-power operating mode; the first correction strategy refers to a strategy for image correction based on a target lookup table; the target lookup table records the pixel value mapping relationship between the image to be corrected and the corrected image obtained by the smart device under the cover material properties.

[0146] In an exemplary embodiment, the apparatus further includes a first selection unit, configured to select a first lookup table and a second lookup table from a lookup table template library that match the material properties of the cover plate; the first lookup table is associated with the first material property; the second lookup table is associated with the second material property; determine a first absolute value of the difference between the second material property and the cover plate material property, and determine a second absolute value of the difference between the first material property and the second material property; determine the ratio between the first absolute value of the difference and the second absolute value of the difference as a first interpolation ratio of the first lookup table; and perform a complement operation on the first interpolation ratio to obtain a second interpolation ratio of the second lookup table. Example: The sum of the first interpolation ratio and the second interpolation ratio is 1; create an empty table, determine each color channel in the empty table as the current color channel, and perform the following interpolation operations to obtain the target lookup table: extract the first pixel value corresponding to the color channel that is the same as the current color channel from the first lookup table, and extract the second pixel value corresponding to the color channel that is the same as the current color channel from the second lookup table; determine the first product between the first interpolation ratio and the first pixel value, and determine the second product between the second interpolation ratio and the second pixel value; determine the sum of the first product and the second product as the target pixel value corresponding to the current color channel.

[0147] In an exemplary embodiment, the device operating state includes the number of image recognition failures of the smart device; the execution unit is configured to respond to the current operating state of the smart device being a complex operating state, and to determine the second correction strategy among multiple correction strategies as the target correction strategy matching the complex operating state; the complex operating state refers to the operating state where the number of image recognition failures is greater than a preset threshold, or the operating state where the material properties of the cover plate are greater than a preset property threshold; the second correction strategy refers to the strategy of performing image correction on the image to be corrected through an image correction model; the image correction model is used to perform color compensation on the image to be corrected and output the corrected image.

[0148] In an exemplary embodiment, the device operating state further includes a version identifier of the image correction model; and an execution unit configured to, before performing image correction on the image to be corrected using the target correction strategy, extract features from the image to be corrected in response to the version identifier of the image correction model being an initial identifier, obtain an optical feature vector, input the optical feature vector to a pre-trained conditional encoder to obtain initial model parameters of the image correction model, and configure the model parameters of the image correction model according to the initial model parameters; and continue to execute the step of performing image correction on the image to be corrected using the target correction strategy in response to the version identifier of the image correction model not being an initial identifier, or the initial model parameters of the image correction model being configured.

[0149] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0150] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0151] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0152] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the method embodiments described above via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0153] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0154] According to another aspect of the embodiments of this application, a computer program product is also provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs various functions provided in the embodiments of this application. The sequence numbers of the embodiments of this application above are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0155] Figure 6 A schematic block diagram of a computer system architecture for implementing embodiments of the present application is shown. Figure 6As shown, the computer system 600 includes a Central Processing Unit (CPU) 601, which performs various appropriate actions and processes based on programs stored in ROM 602 or loaded into RAM 603 from storage section 608. Random access memory 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0156] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card, such as a local area network card or modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0157] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs various functions defined in the system of this application.

[0158] It should be noted that, Figure 6 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0159] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0160] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. An image correction method for a smart device, characterized in that, include: In response to an image acquisition command, the image to be corrected from the smart device and the device operating status of the smart device are acquired; The current operating status of the smart device is determined based on the device's operating status and the cover plate's material properties; the cover plate's material properties are the material properties of the smart device's cover plate; the cover plate's material properties are obtained in response to the image acquisition command or periodically according to a preset acquisition cycle; the cover plate's material properties dynamically change with the physical degradation of the cover plate. A target correction strategy matching the current operating condition is determined from multiple correction strategies, and the target correction strategy is used to correct the image to be corrected to obtain a corrected image, wherein the corrected image is used for image recognition.

2. The method according to claim 1, characterized in that, The intelligent device includes a visible light camera, a near-infrared illumination unit, and a near-infrared camera; the method further includes: Visible light images are acquired by the visible light camera, and reflected images are simultaneously acquired by the near-infrared camera when the near-infrared illumination unit emits light signals of the first and second wavelengths, thus obtaining the first and second near-infrared images. Optical feature vectors are extracted from the visible light image, the first near-infrared image, and the second near-infrared image, and the material properties of the cover plate are determined based on the optical feature vectors.

3. The method according to claim 2, characterized in that, The smart device further includes an ambient light sensor and a temperature sensor. The ambient light sensor is used to collect the ambient light intensity of the smart device; the temperature sensor is used to collect the current temperature of the smart device; the step of extracting optical feature vectors based on the visible light image, the first near-infrared image, and the second near-infrared image includes: Determine at least one of the following optical characteristics: the average reflectance of the first near-infrared image, the average reflectance of the second near-infrared image, the ratio of the reflectance of the first near-infrared image to that of the second near-infrared image, the signal-to-noise ratio of the first near-infrared image, the signal-to-noise ratio of the second near-infrared image, the average pixel values ​​of the red, green, and blue channels in the visible light image, the texture sharpness index of the visible light image, the proportion of specular highlight area in the visible light image, and the image alignment quality score between the visible light image, the first near-infrared image, and the second near-infrared image; the image alignment quality score is used for the spatial registration accuracy of the visible light image, the first near-infrared image, and the second near-infrared image. The optical feature vector is obtained by fusing the at least one optical feature.

4. The method according to claim 1, characterized in that, The device operating status includes the current battery level and system operating mode of the smart device; determining the target correction strategy that matches the current operating status from multiple correction strategies includes: In response to the current operating state of the smart device being a power-saving operating state, the first correction strategy among the multiple correction strategies is determined as the target correction strategy matching the power-saving operating state; the power-saving operating state refers to the operating state where the current battery power of the smart device is lower than a preset power threshold, or the operating mode of the system is in a low-power operating mode; the first correction strategy refers to a strategy for image correction based on a target lookup table; the target lookup table records the pixel value mapping relationship between the image to be corrected and the corrected image obtained by the smart device under the cover material properties.

5. The method according to claim 4, characterized in that, The method further includes: Select a first lookup table and a second lookup table from the lookup table template library that match the material attribute of the cover plate; the first lookup table is associated with the first material attribute; the second lookup table is associated with the second material attribute; The absolute value of the first difference between the second material attribute and the cover plate material attribute is determined, and the absolute value of the second difference between the first material attribute and the second material attribute is determined; the ratio between the absolute value of the first difference and the absolute value of the second difference is determined as the first interpolation ratio of the first lookup table; the complement operation is performed on the first interpolation ratio to obtain the second interpolation ratio of the second lookup table; the sum of the first interpolation ratio and the second interpolation ratio is 1; Create an empty table, and determine each color channel in the empty table as the current color channel. Perform the following interpolation operation to obtain the target lookup table: extract the first pixel value corresponding to the color channel that is the same as the current color channel from the first lookup table, and extract the second pixel value corresponding to the color channel that is the same as the current color channel from the second lookup table; determine the first product between the first interpolation ratio and the first pixel value, and determine the second product between the second interpolation ratio and the second pixel value; and determine the sum of the first product and the second product as the target pixel value corresponding to the current color channel.

6. The method according to claim 1, characterized in that, The device operating status includes the number of image recognition failures of the smart device; determining the target correction strategy that matches the current operating status from multiple correction strategies includes: In response to the current operating state of the smart device being a complex operating condition, the second correction strategy among the multiple correction strategies is determined as the target correction strategy matching the complex operating condition; the complex operating condition refers to the operating condition where the number of image recognition failures exceeds a preset threshold, or the operating condition where the material properties of the cover plate exceed a preset attribute threshold; the second correction strategy refers to the strategy of performing image correction on the image to be corrected through an image correction model; the image correction model is used to perform color compensation on the image to be corrected and output the corrected image.

7. The method according to claim 6, characterized in that, The device operating status also includes the version identifier of the image correction model; before applying the target correction strategy to perform image correction on the image to be corrected, the method further includes: In response to the version identifier of the image correction model being the initial identifier, feature extraction is performed on the image to be corrected to obtain an optical feature vector. The optical feature vector is then input into a pre-trained conditional encoder to obtain the initial model parameters of the image correction model. The model parameters of the image correction model are then configured according to the initial model parameters. If the version identifier of the image correction model is not the initial identifier, or the initial model parameter configuration of the image correction model is completed, the step of performing image correction on the image to be corrected using the target correction strategy continues.

8. An image correction device for a smart device, characterized in that, include: The acquisition unit is used to acquire the image to be corrected from the smart device and the device operating status of the smart device in response to the image acquisition command; The determining unit is used to determine the current operating status of the smart device based on the device's operating status and the cover material properties; the cover material properties are the material properties of the cover of the smart device; the cover material properties are obtained by detection in response to the image acquisition command or by periodic detection according to a preset acquisition cycle; the cover material properties change dynamically with the physical degradation of the cover. An execution unit is configured to determine a target correction strategy that matches the current operating condition from a variety of correction strategies, and to perform image correction on the image to be corrected using the target correction strategy to obtain a corrected image, wherein the corrected image is used for image recognition.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images