Method for calibrating image quality of visual sensor by using related sensor

Abstract

The invention relates to a method for calibrating the image quality of a visual sensor by using a related sensor, and belongs to the technical field of image processing, the method for calibrating the image quality of the visual sensor by using the related sensor comprises the following steps: S1) carrying out quality evaluation on an image acquired by the visual sensor; s2) using a deep learning algorithm to train a long short-term memory (LSTM) network model by analyzing a correlation relationship between related sensor data and image feature quality; s3) predicting image feature quality by using an LSTM model; and S4) adjusting visual sensor parameters according to the image feature quality to optimize the image quality. The adaptive capacity of the visual sensor to a complex shooting environment is enhanced, and standardization and automation of parameter adjustment are realized through a function relation formula.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular, relates to a method for calibrating the image quality of a visual sensor by using a related sensor. Background Art

[0002] As the Internet of Things technology becomes more mature, more and more sensor devices are installed in important positions in various industries. Among them, visual sensors such as cameras and video cameras (hereinafter referred to as visual sensors) are relatively basic, intuitive and simple monitoring methods. On important structures, structural health monitoring relies on massive, multi-type, and relatively high-density sensor Internet of Things networks. The data of this type of network that constitutes the structural health monitoring system usually includes structural parameters such as vibration acceleration, displacement, inclination, GNSS positioning, deflection, structural temperature and humidity, as well as environmental parameters such as wind speed and direction, rainfall, temperature and humidity, air quality, visibility, light, etc., and these sensors are usually distributed inside and outside the structure, with the characteristics of wide coverage, rich content, and high collection accuracy. For example, in the mainstream bridge structure health monitoring system, the vibration acceleration sensor of the cable can collect measurement point acceleration data at a frequency of 120Hz, and the collected data can reflect the vibration frequency and amplitude of the object being photographed with high accuracy. Visual sensors are also common structural monitoring sensors. For example, JT_T1037-2022 "Technical Specifications for Highway Bridge Structure Monitoring" can use visual sensors to monitor displacement, inclination, deflection, traffic load and other data.

[0003] However, existing methods for optimizing image quality of visual sensors mainly rely on human experience and manual parameter adjustment, which makes it difficult to actively adapt to complex and changing environments and shooting scenes. Traditional image parameter settings are usually static and cannot be adjusted in real time according to environmental changes and the state of the object being photographed, resulting in image quality degradation in complex environments such as insufficient light and severe vibration, and even failing to meet shooting requirements. In addition, existing methods focus more on the impact of a single environmental factor (such as light intensity) and lack comprehensive consideration of multi-dimensional environmental information. For example, the impact of factors such as vibration, temperature and humidity on image quality is usually ignored, resulting in poor system robustness. Summary of the invention

[0004] In order to solve the technical problem that the image quality of existing visual sensors is limited by environmental factors, resulting in low image quality and inability to adjust in real time, the present invention provides a method for calibrating the image quality of visual sensors using relevant sensors, and an image quality adaptive optimization method based on deep learning to achieve dynamic adaptive optimization of image quality, enhance the adaptability of visual sensors to complex shooting environments, and achieve standardization and automation of parameter adjustment through functional relationship formulas.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A method for calibrating the image quality of a visual sensor using a correlation sensor comprises the following steps:

[0007] S1) performing quality assessment on the image collected by the visual sensor to determine the image feature quality; the quality assessment includes a blur detection method, a noise detection method, or a combination thereof; the image feature quality includes a dynamic blur degree and image clarity;

[0008] Preferably, the quality assessment process of the fuzzy detection method includes:

[0009] Image preprocessing: convert color images into grayscale images; use Gaussian filtering method to denoise the image to reduce the impact of noise on blur detection;

[0010] Calculate the variance of the Laplacian operator: Apply the Laplacian operator to convolve the grayscale image to obtain the second-order derivative of the image; calculate the variance of the Laplacian operator result based on the second-order derivative of the image. The larger the variance, the clearer the image; the smaller the variance, the blurrier the image;

[0011] Fast Fourier Transform (FFT): Perform a fast Fourier transform on a grayscale image to obtain a frequency domain representation of the image;

[0012] According to the frequency domain representation of the image, the interpolation of high-frequency and low-frequency components is analyzed; the more high-frequency components, the clearer the image; the fewer high-frequency components, the blurrier the image.

[0013] Determine the degree of motion blur: Determine the degree of motion blur of the image based on the calculated variance and high and low frequency interpolation results.

[0014] Preferably, the quality assessment process of the noise detection method includes:

[0015] Image grayscale and denoising;

[0016] The local extreme value noise detection method is used to detect the maximum and minimum pixel values ​​in the local area of ​​the image.

[0017] If a pixel value is a local maximum or minimum value and is significantly different from surrounding pixels, it is considered a noise point.

[0018] Count the number of noise points in the image;

[0019] Analyze the distribution of noise points in the image, and then calculate the number of noise points per unit area to obtain the noise point density;

[0020] According to the noise density, the clarity of the image is evaluated; the higher the noise density, the lower the image clarity. Preferably, the degree of dynamic blur or image clarity is divided into digital grades using manual annotation;

[0021] S2) Using a deep learning algorithm, a long short-term memory network (LSTM) model is trained by analyzing the correlation between the relevant sensor data and the image feature quality; the specific steps include:

[0022] S21) aligning the data collection time of all sensors to ensure that the data collected by all sensors are synchronized;

[0023] S22) collecting images from a visual sensor and extracting time series data of image feature quality, including a degree of dynamic blur and image clarity;

[0024] S23) collecting time series data of relevant sensors, including vibration acceleration, light intensity, and temperature and humidity;

[0025] S24) Clean, normalize and denoise the data to ensure data quality.

[0026] S25) constructing an LSTM model structure, including an input layer, an LSTM layer and an output layer; wherein the input is the relevant sensor data of the time series and the image feature quality;

[0027] Choose appropriate activation functions (such as tanh, sigmoid) and loss functions (such as mean squared error);

[0028] S26) using the training data to perform forward propagation and back propagation, so that the model learns the correlation between the relevant sensor data and the image feature quality, and optimizes the model parameters;

[0029] Adjust model hyperparameters through cross-validation to prevent overfitting;

[0030] S27) Use test data to evaluate model performance indicators, including accuracy, precision, and recall;

[0031] The LSTM model is further optimized based on the evaluation results.

[0032] Preferably, the related sensors include a vibration acceleration sensor, a light intensity sensor, and a temperature and humidity sensor.

[0033] S3) Apply LSTM model to predict image feature quality:

[0034] Input the latest collected relevant sensor data into the trained LSTM model for real-time analysis and prediction of image feature quality;

[0035] S4) adjusting visual sensor parameters and auxiliary functions according to the image feature quality to optimize the image quality;

[0036] Wherein, each digital level of the degree of dynamic blur or image clarity corresponds to a unit adjustment amount of a reasonable visual sensor parameter; the visual sensor parameter includes a shutter speed and an aperture;

[0037] When the degree of motion blur exceeds the digital level threshold, a reasonable unit adjustment is performed on the shutter speed of the visual sensor;

[0038] When the image sharpness exceeds the digital grade threshold, a reasonable unit adjustment is performed on the vision sensor's aperture.

[0039] Preferably, the image clarity is also used to determine whether the subject is underexposed, and to turn on or off the auxiliary function of the visual sensor.

[0040] Preferably, the degree of dynamic blur is used to determine whether the subject is vibrating or moving at high speed, and is used to execute the opening and closing of the auxiliary function of the visual sensor.

[0041] Preferably, the auxiliary functions include fill light and anti-shake functions.

[0042] Preferably, the method further comprises step S5) establishing a functional relationship formula:

[0043] The functional relationship formula between the image feature quality and the unit adjustment amount of the corresponding visual sensor parameters is established, where the functional relationship formula between the degree of dynamic blur and the shutter speed is expressed as:

[0044] B = kS + b;

[0045] In the formula, B represents the degree of motion blur; S represents the shutter speed; K is the proportional constant, and b is a constant;

[0046] The functional relationship between image clarity and aperture is expressed as:

[0047] C = a1n(X) + m;

[0048] In the formula, C represents the image clarity; X represents the aperture; a represents the coefficient of influence of logarithmic change on image clarity; m represents the constant term;

[0049] When the image feature quality exceeds a threshold, a unit adjustment amount of the corresponding visual sensor parameter is automatically performed according to the function value.

[0050] Preferably, the method further comprises step S6): bringing the predicted image feature quality into the corresponding functional relationship formula to calculate the function value, and then adjusting the visual sensor parameters in real time.

[0051] Beneficial effects of the present invention:

[0052] 1. The core technical concept of the present invention is to establish an automatic optimization scheme based on data-driven. First, the classic image processing technology is used to evaluate the image quality; then, through the correlation between the relevant sensor data and the image feature quality, the LSTM neural network algorithm is constructed to predict the image feature quality of the visual sensor in real time; and dynamically adjust the visual sensor parameters (shutter speed, aperture) and auxiliary functions (fill light, anti-shake). Different from the traditional static image parameter settings, the system can adjust the parameters in real time according to various types of external environmental changes (vibration, light, temperature and humidity), and realize dynamic adaptive optimization of image quality. This enables the system to maintain good imaging effects in various complex environments.

[0053] 2. Starting from conventional environmental factors, the present invention introduces sensor data such as vibration acceleration and light intensity to build a more comprehensive environmental perception capability. By learning the relationship between these multi-dimensional sensor data and image feature quality through the LSTM model, the system can more accurately predict image quality and make corresponding parameter adjustments and switch control of auxiliary functions for various complex scenes (such as strong vibration, low light, weather changes, etc.). It greatly improves the system's adaptability to complex shooting environments, ensures that high-quality images can be obtained in various situations, enhances the robustness of image quality, and expands application scenarios.

[0054] 3. Through the functional relationship formula between the image feature quality and the corresponding unit adjustment amount of the visual sensor parameter, standardized and automated calculation and execution of the adjustment of the visual sensor parameters (shutter speed, aperture) and auxiliary functions (fill light, anti-shake) are achieved, thereby achieving fully automatic real-time optimization of the monitoring image of the visual sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0056] Figure 1 A schematic diagram of the design framework principle of a method for calibrating the image quality of a visual sensor using a correlation sensor according to the present invention.

[0057] Figure 2 The present invention is a flow chart of a method for calibrating the image quality of a visual sensor using a correlation sensor. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] See also Figure 1-Figure 2 As shown, a method for calibrating the image quality of a visual sensor using a correlation sensor, a method for calibrating the image quality of a visual sensor using a correlation sensor, comprising the following steps:

[0060] S1) performing quality assessment on the image collected by the visual sensor to determine the image feature quality; the quality assessment includes a blur detection method, a noise detection method, or a combination thereof; the image feature quality includes a dynamic blur degree and image clarity;

[0061] Furthermore, the quality assessment process of the fuzzy detection method includes:

[0062] Image preprocessing: convert color images into grayscale images; use Gaussian filtering method to denoise the image to reduce the impact of noise on blur detection;

[0063] Calculate the variance of the Laplacian operator: Apply the Laplacian operator to convolve the grayscale image to obtain the second-order derivative of the image; calculate the variance of the Laplacian operator result based on the second-order derivative of the image. The larger the variance, the clearer the image; the smaller the variance, the blurrier the image;

[0064] Fast Fourier Transform (FFT): Perform a fast Fourier transform on a grayscale image to obtain a frequency domain representation of the image;

[0065] According to the frequency domain representation of the image, the interpolation of high-frequency and low-frequency components is analyzed; the more high-frequency components, the clearer the image; the fewer high-frequency components, the blurrier the image.

[0066] Determine the degree of motion blur: Determine the degree of motion blur of the image based on the calculated variance and high and low frequency interpolation results.

[0067] Specifically, these two indicators can be weighted averaged or other fusion strategies can be used to obtain a comprehensive blur score. This score can be mapped to a predefined numerical level (e.g., 1-10, where 1 represents the best quality and 10 represents the worst quality) to facilitate subsequent processing.

[0068] Furthermore, the quality assessment process of the noise detection method includes:

[0069] Image grayscale and denoising;

[0070] The local extreme value noise detection method is used to detect the maximum and minimum pixel values ​​in the local area of ​​the image.

[0071] If a pixel value is a local maximum or minimum value and is significantly different from surrounding pixels, it is considered a noise point.

[0072] Count the number of noise points in the image;

[0073] Analyze the distribution of noise points in the image, and then calculate the number of noise points per unit area to obtain the noise point density;

[0074] Evaluates the clarity of the image based on the noise density; the higher the noise density, the lower the image clarity.

[0075] It should be noted that the image feature quality (dynamic blur and image clarity) generated in step S1) is the key data for LSTM model training. These data, together with the relevant sensor data collected at the same time, constitute the training samples of the LSTM model.

[0076] Dynamic blur: refers to the degree of blur caused by moving objects in the image. It reflects the degree of image blur caused by the object moving too fast or the camera shaking during the image acquisition process. The higher the dynamic blur, the blurrier the image, the more details are lost, and the worse the image quality. It is closely related to the camera's shutter speed, anti-shake function, and the speed of the object being photographed.

[0077] Image clarity: refers to the clarity and sharpness of image details. It reflects the sharpness of details in the image and whether there are factors such as noise and artifacts in the image that affect detail clarity. The higher the image clarity, the clearer the details and the better the image quality. It is closely related to the camera's aperture, lens quality and the light conditions of the imaging environment.

[0078] The above two image features are mainly representative, but the present invention is not limited thereto, and the following image features may also be used:

[0079] Color reproduction: refers to whether the color of the image accurately reproduces the color of the real scene. The quality of color reproduction directly affects the realism and visual effect of the image.

[0080] Image contrast: refers to the difference between the light and dark areas in an image. Too low a contrast will result in a dark image with a lack of depth; too high a contrast will result in a harsh image with lost details. Appropriate contrast can enhance the depth and visual impact of an image.

[0081] Exposure: refers to the brightness and contrast of an image. Underexposure will result in a dark image with lost details; overexposure will result in an overly bright image with lost details and possible overexposed areas.

[0082] As a preferred implementation, by comprehensively considering the above multiple image features, the image quality can be evaluated more comprehensively and objectively, and appropriate image quality evaluation indicators can be selected according to specific application scenarios. This transforms the traditional image quality optimization method that relies on human experience and manual parameter adjustment into a data-driven, automated, intelligent control method.

[0083] Specifically, the degree of dynamic blur or image clarity is divided into digital grades using image automated analysis tools or manual annotation. Specifically, it is necessary to rate the degree of dynamic blur and image clarity of each image according to a predefined grade standard, and assign a corresponding grade label to each image. The quality of the annotated data directly affects the training effect of the LSTM model and the final image quality optimization effect.

[0084] Therefore, sufficient manpower and material resources need to be invested to ensure the quality and quantity of the labeled data. In addition, repeated and rotated labeling of the same image can be considered, and statistical or other methods can be used to improve the reliability of labeling.

[0085] S2) Using a deep learning algorithm, a long short-term memory network (LSTM) model is trained by analyzing the correlation between the relevant sensor data and the image feature quality; the specific steps include:

[0086] S21) aligning the data collection time of all sensors to ensure that the data collected by all sensors are synchronized;

[0087] S22) collecting images from a visual sensor and extracting time series data of image feature quality, including a degree of dynamic blur and image clarity;

[0088] S23) collecting time series data of relevant sensors, including vibration acceleration, light intensity, and temperature and humidity;

[0089] S24) Clean, normalize and denoise the data to ensure data quality.

[0090] S25) constructing an LSTM model structure, including an input layer, an LSTM layer and an output layer; wherein the input is the relevant sensor data of the time series and the image feature quality;

[0091] Choose appropriate activation functions (such as tanh, sigmoid) and loss functions (such as mean squared error);

[0092] S26) using the training data to perform forward propagation and back propagation, so that the model learns the correlation between the relevant sensor data and the image feature quality, and optimizes the model parameters;

[0093] Adjust model hyperparameters through cross-validation to prevent overfitting;

[0094] S27) Use the test data to evaluate model performance indicators, including accuracy, precision, and recall;

[0095] The LSTM model is further optimized based on the evaluation results.

[0096] Furthermore, the related sensors include a vibration acceleration sensor, a light intensity sensor, and a temperature and humidity sensor.

[0097] Specifically, the LSTM model is good at processing time series data: the impact of environmental factors (light, vibration, etc.) on image quality is usually not isolated, but changes over time. LSTM is good at processing time series data, can capture these time dependencies, and predict future image quality based on historical data. Traditional machine learning methods, such as support vector machines (SVM) or random forests, have difficulty in effectively processing such time series data.

[0098] Among them, the impact of environmental factors on image quality may have long-term dependencies. For example, continuous strong light may cause the sensor to overheat, thus affecting image quality; continuous severe vibration may also cause long-term image blur. LSTM can effectively capture these long-term dependencies, while traditional recurrent neural networks (RNNs) are prone to gradient vanishing problems and have difficulty learning long-term dependencies.

[0099] In addition, the relationship between environmental factors and image quality is usually nonlinear. LSTM can effectively learn this nonlinear relationship, while traditional linear models have difficulty in accurately modeling such complex relationships.

[0100] In summary, the LSTM model predicts the quality of future images based on the learned time dependency, thereby providing a basis for parameter adjustment and laying the foundation for predictive image quality control.

[0101] S3) Apply LSTM model to predict image feature quality:

[0102] Input the latest collected relevant sensor data into the trained LSTM model for real-time analysis and prediction of image feature quality;

[0103] S4) adjusting visual sensor parameters and auxiliary functions according to the image feature quality to optimize the image quality;

[0104] Wherein, each digital level of the degree of dynamic blur or image clarity corresponds to a unit adjustment amount of a reasonable visual sensor parameter; the visual sensor parameter includes a shutter speed and an aperture;

[0105] When the degree of motion blur exceeds the digital level threshold, a reasonable unit adjustment is performed on the shutter speed of the visual sensor;

[0106] When the image sharpness exceeds the digital grade threshold, a reasonable unit adjustment is performed on the vision sensor's aperture.

[0107] Furthermore, the image clarity is also used to determine whether the subject is underexposed and to turn the visual sensor auxiliary function on and off.

[0108] Furthermore, the degree of dynamic blur is used to determine whether the subject is vibrating or moving at high speed, and is used to turn on and off the auxiliary function of the visual sensor.

[0109] Furthermore, the auxiliary functions include fill light and anti-shake functions.

[0110] Specifically, since there is a correlation between the image feature quality and the unit adjustment amount of the corresponding visual sensor parameters, and the correlation is not necessarily positive, the motion blur threshold is level 4 (that is, when the predicted motion blur level reaches level 4 or above, the shutter speed needs to be adjusted).

[0111] Image clarity threshold: Level 5 (i.e., when the predicted image clarity is less than Level 5 or below, the aperture needs to be adjusted).

[0112] In addition to adjusting shutter speed and aperture, you can also turn auxiliary functions on or off based on the quality of image characteristics:

[0113] Motion blur level ≥ Level 6 (or other preset threshold): turn on the anti-shake function.

[0114] Image clarity ≤ Level 4 (or other preset threshold): turn on the fill light.

[0115] In the specific implementation process, the threshold and unit adjustment amount need to be carefully adjusted according to the specific sensor characteristics, environmental conditions and application requirements, and the adjustment of other parameters such as ISO value and filter, as well as more complex control logic may need to be considered. The unit adjustment amount should not be increased blindly, and the physical limitations of the sensor parameters need to be considered to avoid the parameters exceeding the valid range. In addition, the parameter adjustment strategy may also need to be fine-tuned and optimized based on experience and actual test results.

[0116] When the lighting conditions and environment are stable, reduce the shutter speed, set a reasonable aperture and ISO value, filter, curtain and other settings to improve the quality of image acquisition.

[0117] In poor lighting conditions and environmental stability, increase the shutter speed, adjust the aperture and ISO value to ensure the recognizability of the image.

[0118] Turn on or off auxiliary functions according to specific circumstances, such as turning on wipers, enhancing or supplementing lighting, increasing target luminous power, turning on anti-shake and other functions according to the influence of weather (rain, fog, haze, strong wind).

[0119] The same settings are also made considering the state of the subject, such as vibration, underexposure, occlusion, etc.

[0120] Further, the method further comprises step S5) establishing a functional relationship formula:

[0121] The functional relationship formula between the image feature quality and the unit adjustment amount of the corresponding visual sensor parameters is established, where the functional relationship formula between the degree of dynamic blur and the shutter speed is expressed as:

[0122] B = kS + b;

[0123] In the formula, B represents the degree of motion blur; S represents the shutter speed; K is the proportional constant, and b is a constant;

[0124] The functional relationship between image clarity and aperture is expressed as:

[0125] C = a1n(X) + m;

[0126] In the formula, C represents the image clarity; X represents the aperture; α represents the coefficient of influence of logarithmic change on image clarity; m represents the constant term;

[0127] When the image feature quality exceeds a threshold, a unit adjustment amount of the corresponding visual sensor parameter is automatically performed according to the function value.

[0128] Specifically, the parameters K, α, and b in the formula need to be determined by fitting experimental data. This can be done by collecting a large amount of image data, manually annotating the image's dynamic blur and image clarity, or automatically evaluating them using the algorithm in step S1, while recording the corresponding shutter speed and aperture value.

[0129] Then, the collected data is used to fit the above two functions using a suitable curve fitting method (such as the least square method, the nonlinear least square method, etc.), so as to determine the values ​​of the parameters K, α, and b.

[0130] On the other hand, the choice of function is not fixed. A more appropriate function form can be selected based on experimental data. It is also possible to consider combining other machine learning methods, such as neural networks, to fit more complex functional relationships.

[0131] Furthermore, the method further includes step S6): bringing the predicted image feature quality into the corresponding functional relationship formula to calculate the function value, and then adjusting the visual sensor parameters and auxiliary functions in real time.

[0132] The present invention proposes an automatic optimization scheme for the image quality of a visual sensor based on deep learning. First, the quality of images collected by the visual sensor is evaluated by an image processing algorithm to quantify the dynamic blur and clarity of the image. Then, relevant sensor data is collected, and a LSTM model is used to learn the time series relationship between the relevant sensor data and image quality characteristics to establish a prediction model. Next, the trained LSTM model is used to predict the image quality in real time. Finally, according to the prediction results, the visual sensor parameters (such as shutter speed, aperture) and auxiliary functions are automatically calculated and adjusted by a pre-established functional relationship formula to optimize the image quality.

[0133] This optimization mechanism realizes dynamic adaptive control of image quality. This solution can significantly improve the stability and reliability of image quality, reduce manual intervention, improve efficiency and automation, and enhance the system's adaptability to complex shooting scenes, ultimately achieving more robust and efficient intelligent image acquisition.

[0134] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the methods, processes and steps described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed methods, processes and functional relationships can be implemented in other ways. In addition, each process in each embodiment of the present application can be integrated into a processing flow, or each process can exist physically separately, or two or more processes can be integrated into one process. The above-mentioned integrated process can be implemented in the form of hardware or in the form of software functional flow.

[0136] If the process is implemented in the form of a software function and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

[0137] In the description of the specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0138] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for calibrating the image quality of a visual sensor using a correlation sensor, characterized in that: The following steps are involved: S1) performing quality assessment on the image collected by the visual sensor to determine the quality of image features; The quality assessment includes a blur detection method, a noise detection method or a combination thereof; the image feature quality includes a dynamic blur degree and an image clarity; S2) using a deep learning algorithm to train an LSTM model by analyzing the correlation between relevant sensor data and image feature quality; S3) Apply LSTM model to predict image feature quality: Input the latest collected relevant sensor data into the trained LSTM model for real-time analysis and prediction of image feature quality; S4) adjusting visual sensor parameters and auxiliary functions according to the image feature quality to optimize the image quality.

2. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 1, characterized in that: The quality assessment process of the fuzzy detection method includes: Image preprocessing: convert color images into grayscale images; use Gaussian filtering method to denoise the image; Calculate the variance of the Laplacian operator: Apply the Laplacian operator to convolve the grayscale image to obtain the second-order derivative of the image; calculate the variance of the Laplacian operator result based on the second-order derivative of the image. The larger the variance, the clearer the image; the smaller the variance, the blurrier the image; Fast Fourier Transform: Perform fast Fourier transform on the grayscale image to obtain the frequency domain representation of the image; According to the frequency domain representation of the image, the interpolation of high-frequency and low-frequency components is analyzed; the more high-frequency components, the clearer the image; the fewer high-frequency components, the blurrier the image; Determine the degree of motion blur: Determine the degree of motion blur of the image based on the calculated variance and high and low frequency interpolation results.

3. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 1, characterized in that: The quality assessment process of the noise detection method includes: Image grayscale and denoising; The local extreme value noise detection method is used to detect the maximum and minimum pixel values ​​in the local area of ​​the image. If a pixel value is a local maximum or minimum value and is significantly different from surrounding pixels, it is considered a noise point. Count the number of noise points in the image; Analyze the distribution of noise points in the image, and then calculate the number of noise points per unit area to obtain the noise point density; Evaluates the clarity of an image based on the noise density; the higher the noise density, the lower the image clarity.

4. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 1, characterized in that: The degree of dynamic blur or image clarity is manually marked and divided into digital levels. The specific process of step S2) includes: S21) aligning the data collection time of all sensors to ensure that the data collected by all sensors are synchronized; S22) collecting images from a visual sensor and extracting time series data of image feature quality, including a degree of dynamic blur and image clarity; S23) collecting time series data of relevant sensors, including vibration acceleration, light intensity, and temperature and humidity; S24) cleaning, normalizing and denoising the data to ensure data quality; S25) constructing an LSTM model structure, including an input layer, an LSTM layer and an output layer; wherein the input is the relevant sensor data of the time series and the image feature quality; and selecting an appropriate activation function and loss function; S26) using the training data for forward propagation and back propagation, so that the model learns the correlation between the relevant sensor data and the image feature quality, and optimizes the model parameters; adjusting the model hyperparameters through cross-validation to prevent overfitting; S27) Use the test data to evaluate the model performance indicators, including accuracy, precision, and recall; and further optimize the LSTM model based on the evaluation results.

5. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 4, characterized in that: The related sensors include vibration acceleration sensors, light intensity sensors and temperature and humidity sensors.

6. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 1, characterized in that: The specific process of step S4) includes: Each numerical level of the degree of motion blur or image clarity corresponds to a unit adjustment of a reasonable visual sensor parameter; the visual sensor parameters include shutter speed and aperture; When the degree of motion blur exceeds the digital level threshold, a reasonable unit adjustment is performed on the shutter speed of the visual sensor; When the image sharpness exceeds the digital grade threshold, a reasonable unit adjustment is performed on the vision sensor's aperture.

7. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 1, characterized in that: Image clarity is also used to determine whether the subject is underexposed and to turn the visual sensor auxiliary function on and off; The degree of motion blur is used to determine whether the subject is vibrating or moving at high speed, and is used to turn the visual sensor auxiliary function on and off.

8. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 7, characterized in that: The auxiliary functions include fill light and anti-shake function.

9. A method for calibrating the image quality of a visual sensor using a correlation sensor according to any one of claims 1 to 6, characterized in that: The step S5) also includes establishing a functional relationship formula: The functional relationship formula between the image feature quality and the unit adjustment amount of the corresponding visual sensor parameters is established, where the functional relationship formula between the degree of dynamic blur and the shutter speed is expressed as: B = kS + b; In the formula, B represents the degree of motion blur; S represents the shutter speed; K is the proportional constant, and b is a constant; The functional relationship between image clarity and aperture is expressed as: C = a1n(X) + m; In the formula, C represents the image clarity; X represents the aperture; a represents the coefficient of influence of logarithmic change on image clarity; m represents the constant term; When the image feature quality exceeds a threshold, a unit adjustment amount of the corresponding visual sensor parameter is automatically performed according to the function value.

10. The method for calibrating the image quality of a visual sensor using a correlation sensor according to claim 9, characterized in that: The method also includes step S6): bringing the predicted image feature quality into the corresponding functional relationship formula to calculate the function value, and then adjusting the visual sensor parameters in real time.

Images