High-precision environment illumination measurement method and system based on deep learning and automatic calibration

Abstract

The invention belongs to the technical field of measurement, and particularly relates to a high-precision environment illumination measurement method and system based on deep learning and automatic calibration. Through multi-spectral image fusion, a phase recovery algorithm and a channel attention mechanism, illumination measurement precision is improved, relative errors are controlled at a lower level, high-precision scene requirements are met, equipment parameter self-calibration is realized by means of an automatic calibration module, reference points do not need to be manually arranged, and by combining transfer learning and active learning strategies, the accuracy of illumination measurement is improved. The system deployment process is greatly simplified, the manual data annotation workload is reduced, the data acquisition efficiency is improved, the adaptability to low-illumination and strong-reflection environments is enhanced by utilizing a multispectral dynamic exposure fusion technology and a reflectivity optimization model, the measurement failure is avoided, and the measurement accuracy is improved by adopting an embedded edge computing architecture and a model quantification technology. The measurement performance is ensured, the hardware cost is reduced, and the equipment integration level and the convenience of engineering application are improved through the multispectral camera and a standardized interface.

Description

Technical Field

[0001] The present invention belongs to the field of measurement technology, and specifically relates to a high-precision ambient illumination measurement method and system based on deep learning and automatic calibration. Background Art

[0002] Ambient illuminance measurement is a core technology in fields such as architectural lighting, industrial vision, and medical testing. In scenarios such as medical operating rooms and semiconductor cleanrooms, precise illuminance control directly affects operational safety and product yield. In the field of smart buildings, high-precision illuminance monitoring is the basis for achieving energy optimization and visual comfort. Traditional illuminance measurement mainly relies on photosensors or single-point illuminance meters, but these methods have fundamental flaws. Sensors can only detect the brightness of local scattered light and cannot directly reflect the actual illuminance of the work surface. They are also significantly affected by surface reflectivity and ambient light. With the development of computer vision technology, camera-based illuminance measurement methods are gradually emerging. A patent application with publication number CN113432712B in the prior art discloses a method for measuring ambient illuminance based on a radial basis function neural network, comprising acquiring grayscale image information using a camera, utilizing illuminance measurement data from illuminance sensors placed at reference points in the measured environment, and measuring ambient working surface illuminance data using an illuminometer to generate a data set; and using the data set to train a radial basis function neural network to estimate the illuminance distribution of the illuminated working surface. This method for measuring illuminance distribution solves the current drawbacks of the cumbersome and expensive calibration process of the camera or camera that must be performed when using a camera or camera for illuminance measurement, as well as the need to read parameters such as the exposure time and gain of the camera or camera during the measurement process. The illuminance distribution measurement has low requirements for the camera or camera and is highly versatile, providing great convenience for engineering applications.

[0003] However, there are still some problems in the actual application of the above scheme. The above scheme requires the deployment of camera equipment and illuminance sensors in the measured environment and the selection of reference points. The equipment layout process is relatively cumbersome. When establishing a training data set, it is necessary to measure the grayscale value and illuminance value of different observation points multiple times to generate a single-valued function of the image variable. The data collection and processing process is time-consuming and labor-intensive. In the test of the above patent embodiment, the relative error between the network output illuminance and the measured illuminance is mostly maintained within 8%. Although the human eye is not sensitive to illuminance changes within 8%, for scenarios with extremely high precision requirements such as medical treatment and scientific research, this error range may not meet the needs, limiting its application in high-precision fields.

[0004] Therefore, there is an urgent need for an innovative high-precision ambient illumination measurement method and system based on deep learning and automatic calibration. Summary of the Invention

[0005] The present invention proposes a high-precision ambient illumination measurement method and system based on deep learning and automatic calibration. The purpose of the present invention is to provide an ambient illumination measurement method and system that can achieve precision breakthroughs, system simplification, data acquisition automation and hardware integration.

[0006] The technical solution of the present invention is as follows: A high-precision ambient illumination measurement method based on deep learning and automatic calibration, comprising the following steps: S1. Obtain multispectral images of the measured environment and the corresponding true illumination data to construct a training data set; S2. A deep learning model is trained using a supervised learning framework based on a physical model. The framework includes a phase recovery module, eliminates illumination scale differences through the Gerchberg-Saxton algorithm, and optimizes network convergence by combining a sinusoidal-to-sinusoidal (S2S) mapping paradigm. The S2S mapping achieves frequency domain feature alignment through Fourier transform. S3. Input the multispectral image collected in real time into the trained model and output high-precision illumination values, with the relative error controlled within 3%.

[0007] Furthermore, the multispectral image contains at least visible light, near infrared and ultraviolet band data, and the accuracy of illumination feature extraction is improved through multi-sensor fusion. The fusion process uses a weighted average function combined with logarithmic transformation to optimize the dynamic range.

[0008] Furthermore, the deep learning model is a hybrid architecture combining the residual network ResNet and the squeeze excitation SE module, wherein the SE module adaptively adjusts the feature weights through the channel attention function, and the formula is F ′= F·s ( W 2 d ( W 1 Favg )); in, F is the input feature map, W 1 and W 2 is the weight of the fully connected layer, d is the ReLU activation function, s is the Sigmoid function.

[0009] Furthermore, the construction of the training dataset adopts a transfer learning optimization strategy: Using the convolutional layer output of the pre-trained model as the feature vector, the formula is f feat =ResNet( I ) conv 5, among which I For the input image, ResNet( I ) conv 5 is the output of the fifth convolutional block of ResNet; The key observation points are selected through active learning strategy, and the model parameters are dynamically updated using entropy sampling function. The formula is: H ( p )=-∑ c =1 Cp ( c ∣ x )log p ( c ∣ x ),in p ( c ∣ x ) is a sample x Belong to category c probability.

[0010] Furthermore, the pre-trained model completes reflectivity feature extraction on the metal surface dataset, and the target domain model optimizes the illumination mapping relationship in complex reflectivity scenes by linearly fitting continuous multi-frame data.

[0011] A high-precision ambient illumination measurement system based on deep learning and automatic calibration, including: Multispectral camera for collecting environmental images; An embedded computing unit integrating the deep learning model described in any one of claims 1 to 3 to implement real-time illumination calculation; The automatic calibration module completes the self-calibration of device parameters through a checkerboard pattern or a predefined calibration plate, eliminating the need for manual reference point setting.

[0012] Furthermore, the automatic calibration module includes: Image feature extraction unit, which identifies checkerboard corners and calculates sub-pixel coordinates, using the Harris corner detection function combined with the least squares method to optimize positioning accuracy; The parameter optimization unit calculates the camera intrinsic parameter A and distortion coefficient through the homography matrix H based on Zhang Zhengyou calibration method k 1, k 2. The formula is H=λA·[r1 r2 t] ,in [r1 r2] is a 3×2 rotation matrix, t is a 3×1 translation vector, l is the scale factor, and the Levenberg-Marquardt algorithm is used to minimize the reprojection error.

[0013] Furthermore, the multispectral camera supports dynamic exposure adjustment and avoids grayscale oversaturation through multiple exposure fusion technology in low-light scenes. The fusion process adopts Laplace pyramid decomposition and weighted fusion function.

[0014] Furthermore, the embedded computing unit converts floating-point operations into fixed-point operations through model quantization technology, and implements edge computing in combination with low-power GPU or ASIC chips to reduce hardware costs.

[0015] The beneficial effects of the present invention are: This invention significantly improves the illumination measurement accuracy through multispectral image fusion, phase recovery algorithm and channel attention mechanism, controls the relative error to a lower level, effectively meets the application requirements of high-precision scenes, and breaks through the accuracy limitations of existing technologies. The present invention uses an automatic calibration module to achieve self-calibration of equipment parameters, eliminating the need for manual intervention in reference point layout, greatly simplifying the system deployment process. At the same time, it uses transfer learning and active learning strategies to significantly reduce the workload of manual data annotation, improve data collection efficiency, and solve the problems of long deployment time and high labor costs in existing technologies. This invention uses multispectral dynamic exposure fusion technology and reflectivity optimization model to effectively improve adaptability to complex environments such as low illumination and strong reflections, avoid measurement failures caused by changes in lighting conditions, and significantly enhance the robustness of the system in different scenarios. This invention uses an embedded edge computing architecture and model quantization technology to reduce hardware costs and improve device integration while ensuring measurement performance. It also adopts a universal multispectral camera and standardized interfaces to reduce dependence on customized hardware, providing a more universal solution for engineering applications and significantly improving the convenience of actual deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Figure 1 is a flow chart of an embodiment of the present invention; Figure 2 This is a system block diagram of a high-precision ambient illumination measurement system based on deep learning and automatic calibration in the present invention; Figure 3 This is a system block diagram of the automatic calibration module of the present invention.

[0018] In the figure: 1. Multispectral camera; 11. Data synchronization unit; 12. Function calculation unit; 2. Embedded computing unit; 3. Automatic calibration module; 31. Image feature extraction unit; 32. Parameter optimization unit. DETAILED DESCRIPTION

[0019] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0020] Example 1 like Figure 1~3 As shown, this embodiment proposes a high-precision ambient illumination measurement method based on deep learning and automatic calibration, including the following steps: S1. Obtain multispectral images of the measured environment and the corresponding true illumination data to construct a training data set; S2. A supervised learning framework based on a physical model is used to train a deep learning model. The framework includes a phase recovery module, which is applied to the raw image data as a preprocessing stage after the multispectral image input. The module is specifically integrated between the input layer and the feature extraction layer of the deep learning model. The module uses the Gerchberg-Saxton algorithm to eliminate the difference in illumination scale and combines the sine component to sine component (S2S) mapping paradigm to optimize network convergence. The S2S mapping achieves frequency domain feature alignment through Fourier transform. The phase recovery module first performs illumination scale normalization on the input multispectral image, and then outputs it to the subsequent hybrid architecture of ResNet and SE modules for feature extraction, ensuring that frequency domain alignment is completed before feature extraction; S3. Input the multispectral image collected in real time into the trained model and output high-precision illumination values, with the relative error controlled within 3%.

[0021] Multispectral images contain at least visible light, near-infrared and ultraviolet band data. The accuracy of illumination feature extraction is improved through multi-sensor fusion. The fusion process uses a weighted average function combined with logarithmic transformation to optimize the dynamic range.

[0022] The deep learning model is a hybrid architecture that combines the residual network ResNet with the squeeze-excitation SE module, where the SE module adaptively adjusts the feature weights through the channel attention function, and the formula is F ′= F·s ( W 2 d ( W 1 Favg )); in, F is the input feature map, W 1 and W 2 is the weight of the fully connected layer, d is the ReLU activation function, s is the Sigmoid function.

[0023] The construction of the training data set adopts the transfer learning optimization strategy: the convolutional layer output of the pre-trained model is used as the feature vector, and the formula is f feat =ResNet( I ) conv 5, among which I For the input image, ResNet( I ) conv 5 is the output of the fifth convolutional block of ResNet; The key observation points are selected through active learning strategy, and the model parameters are dynamically updated using entropy sampling function. The formula is: H ( p )=-∑ c =1 Cp ( c ∣ x )log p ( c ∣ x ),in p ( c ∣ x ) is a sample x Belong to category c probability.

[0024] The pre-trained model completes reflectivity feature extraction on the metal surface dataset, and the target domain model optimizes the illumination mapping relationship in complex reflectivity scenes by linearly fitting continuous multi-frame data.

[0025] A high-precision ambient illumination measurement system based on deep learning and automatic calibration, including: Multispectral camera for collecting environmental images; An embedded computing unit integrating the deep learning model of any one of claims 1 to 3 to implement real-time illumination calculation; The automatic calibration module completes the self-calibration of device parameters through a checkerboard pattern or a predefined calibration plate, eliminating the need for manual reference point setting.

[0026] The automatic calibration module includes: Image feature extraction unit, which identifies checkerboard corners and calculates sub-pixel coordinates, using the Harris corner detection function combined with the least squares method to optimize positioning accuracy; The parameter optimization unit calculates the camera intrinsic parameter A and distortion coefficient through the homography matrix H based on Zhang Zhengyou calibration method k 1, k 2. The formula is H=λA·[r1 r2 t] ,in [r1 r2] is a 3×2 rotation matrix, t is a 3×1 translation vector, lis the scale factor, and the Levenberg-Marquardt algorithm is used to minimize the reprojection error.

[0027] The multispectral camera supports dynamic exposure adjustment and avoids grayscale oversaturation through multiple exposure fusion technology in low-light scenes. The fusion process uses Laplace pyramid decomposition and weighted fusion function.

[0028] The embedded computing unit converts floating-point operations into fixed-point operations through model quantization technology, and combines it with low-power GPU or ASIC chips to achieve edge computing and reduce hardware costs.

[0029] The present invention is further described in detail below with reference to the embodiments: (1) Illumination measurement in medical operating rooms System deployment: The FLIR A655sc multispectral camera supports acquisition of visible light (400-700nm) and near-infrared (700-1700nm) bands, and is equipped with an ultraviolet filter to expand the 200-400nm band.

[0030] The automatic calibration module uses an 8×8 checkerboard calibration plate. The camera is 1.5 meters away from the calibration plate. 20 sets of images at different angles are collected for internal calibration. The reprojection error is controlled within 0.5 pixels using the Levenberg-Marquardt algorithm.

[0031] Model training: 500 sets of multispectral images were collected under different operating room lighting conditions (400-1500 lux). A Konica Minolta T-10A illuminometer was used to obtain ground truth values ​​and construct an image-illuminance mapping pair. Using the ResNet-50 model pre-trained on the ImageNet dataset, extract the fifth convolutional block features (ResNet (I) conv5) , 200 groups of high uncertainty samples were screened by entropy sampling function for manual annotation, and the remaining 300 groups were automatically expanded through model prediction; In the channel attention function of the SE module, W 1 dimension is C × C / 16,W 2 dimensions are C / 16 × C ,The feature extraction capability of the reflective area of ​​surgical instruments is enhanced through ,adaptive weight adjustment.

[0032] Real-time measurement: The camera captures multispectral images at 30fps, and the embedded computing unit (NVIDIA JetsonAGX Orin) uses model quantization technology (INT8 fixed-point operations) to achieve real-time inference, with a relative error of ≤2.8% in the output illumination value. In low-light scenarios (such as when the surgical light is off), images with three different exposure times (1 / 100s, 1 / 200s, and 1 / 500s) are fused to avoid noise interference, improving fusion accuracy by 15%.

[0033] (2) Semiconductor clean room illumination monitoring System optimization: A polarizer is added to the multispectral camera to eliminate the influence of specular reflection on the wafer surface. The ultraviolet band (254nm) is used to monitor the illumination distribution in the photoresist curing area. The automatic calibration module integrates a motion platform, automatically moving the calibration plate to different positions every hour, dynamically updating the camera's internal parameters to adapt to the changes in viewing angle caused by the movement of the robotic arm in the clean room.

[0034] Accuracy verification: 100 points were tested within the range of 100-1000 lux. The measurement results of the method of the present invention were compared with those of a standard illuminometer (error ±1%). The maximum relative error was 2.5%, and the average relative error was 1.8%, meeting the national standard for the accuracy of clean room illumination measurement.

[0035] It should also be noted that during the model training process, the phase recovery module acts as a preprocessing layer, directly acting on the original multispectral image input, and its output serves as the input feature of the ResNet-50 network. This position selection ensures that the illumination scale difference is eliminated before feature extraction, avoiding frequency domain noise in the subsequent S2S mapping, thereby optimizing convergence.

[0036] The above embodiments show that the present invention effectively solves the accuracy and deployment problems of the existing technology by combining deep learning with automatic calibration technology, and provides an engineering solution for high-precision ambient illumination measurement.

[0037] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-precision ambient illumination measurement method based on deep learning and automatic calibration, characterized in that: The following steps are involved: S1. Obtain multispectral images of the measured environment and the corresponding true illumination data to construct a training data set; S2. A deep learning model is trained using a supervised learning framework based on a physical model. The framework includes a phase recovery module, eliminates illumination scale differences through the Gerchberg-Saxton algorithm, and optimizes network convergence by combining a sinusoidal-to-sinusoidal (S2S) mapping paradigm. The S2S mapping achieves frequency domain feature alignment through Fourier transform. S3. Input the multispectral image collected in real time into the trained model and output high-precision illumination values.

2. The high-precision ambient illumination measurement method based on deep learning and automatic calibration according to claim 1, characterized in that: The multispectral image contains at least visible light, near-infrared and ultraviolet band data, and the accuracy of illumination feature extraction is improved through multi-sensor fusion. The fusion process uses a weighted average function combined with logarithmic transformation to optimize the dynamic range.

3. The high-precision ambient illumination measurement method based on deep learning and automatic calibration according to claim 1, characterized in that: The deep learning model is a hybrid architecture combining the residual network ResNet and the squeeze-excitation SE module, where the SE module adaptively adjusts the feature weights through the channel attention function, and the formula is F ′= F·σ ( W 2 δ ( W 1 Favg )); in, F is the input feature map, W 1 and W 2 is the weight of the fully connected layer, δ is the ReLU activation function, σ is the Sigmoid function.

4. The high-precision ambient illumination measurement method based on deep learning and automatic calibration according to claim 1, characterized in that: The construction of the training dataset adopts the transfer learning optimization strategy: Using the convolutional layer output of the pre-trained model as the feature vector, the formula is f feat =ResNet( I ) conv 5, among which I For the input image, ResNet( I ) conv 5 is the output of the fifth convolutional block of ResNet; The key observation points are selected through active learning strategy, and the model parameters are dynamically updated using entropy sampling function. The formula is: H ( p )=-∑ c =1 Cp ( c ∣ x )log p ( c ∣ x ),in p ( c ∣ x ) is a sample x Belong to category c probability.

5. The high-precision ambient illumination measurement method based on deep learning and automatic calibration according to claim 4 is characterized in that: The pre-trained model completes reflectivity feature extraction on the metal surface dataset, and the target domain model optimizes the illumination mapping relationship in complex reflectivity scenes by linearly fitting continuous multi-frame data.

6. A high-precision ambient illumination measurement system based on deep learning and automatic calibration, characterized in that: include: Multispectral camera for collecting environmental images; An embedded computing unit integrating the deep learning model described in any one of claims 1 to 3 to implement real-time illumination calculation; The automatic calibration module completes the self-calibration of device parameters through a checkerboard pattern or a predefined calibration plate, eliminating the need for manual reference point setting.

7. The high-precision ambient illumination measurement system based on deep learning and automatic calibration according to claim 6, characterized in that: The automatic calibration module comprises: Image feature extraction unit, which identifies checkerboard corners and calculates sub-pixel coordinates, using the Harris corner detection function combined with the least squares method to optimize positioning accuracy; The parameter optimization unit calculates the camera intrinsic parameter A and distortion coefficient through the homography matrix H based on Zhang Zhengyou calibration method k 1, k 2. The formula is H=λA·[r1 r2 t] ,in [r1 r2] is a 3×2 rotation matrix, t is a 3×1 translation vector, λ is the scale factor, and the Levenberg-Marquardt algorithm is used to minimize the reprojection error.

8. The high-precision ambient illumination measurement system based on deep learning and automatic calibration according to claim 6, characterized in that: The multispectral camera supports dynamic exposure adjustment and avoids grayscale oversaturation through multiple exposure fusion technology in low-light scenes. The fusion process adopts Laplace pyramid decomposition and weighted fusion function.

9. The high-precision ambient illumination measurement system based on deep learning and automatic calibration according to claim 6, characterized in that: The embedded computing unit converts floating-point operations into fixed-point operations through model quantization technology, and implements edge computing in combination with a low-power GPU or ASIC chip to reduce hardware costs.

Images