A method for adjusting a camera system under complex conditions of light application

Abstract

The application relates to the field of imaging technology and discloses a camera system adjusting method under complex light application conditions, which comprises an initialization module, an image acquisition module, an image preprocessing module, a pose optimization module, a data verification module, a data statistics module, a data calculation module and a management feedback module. The initialization module is used for calibrating and setting parameters, the image acquisition module is used for acquiring image data, the image preprocessing module is used for preprocessing data, the pose optimization module is used for constructing a pose optimization simulation model, the model adjusts parameters according to a simulation result, optimizes pose parameters, and records optimization data, the data verification module verifies the optimization data of the pose optimization module, the data statistics module classifies and numbers original image data and optimization data, the data calculation module calculates the numbered data of the data statistics module, the calculation result is managed in the management feedback module, and feedback adjustment is performed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of imaging technology, in particular to a camera system adjustment method under complex light application conditions. BACKGROUND

[0002] In a complex lighting environment, the pose estimation of the traditional visual inertial navigation system (VINS) algorithm may be inaccurate. This is because factors such as light and camera exposure can affect image grayscale, thereby affecting the accuracy of visual information and leading to a decline in the quality of pose estimation. This problem is particularly prominent under complex light application conditions, because in these conditions, the environmental perception and adaptive adjustment capabilities of the imaging device are limited, and the analysis of environmental light changes is relatively "rough" and cannot be fully quantitatively described. In addition, in order to fully utilize the advantages of high frame rate of IMU information and meet the real-time frame rate requirements of image flow, mainstream visual inertial SLAM methods mostly use semi-direct method to process image data association problems, which improves real-time performance but may sacrifice accuracy in some cases. Therefore, a camera system adjustment method is needed that can improve the accuracy of pose estimation under complex lighting conditions. SUMMARY

[0003] (1) Technical problems solved

[0004] To overcome the shortcomings of the prior art, the present application provides a camera system adjustment method under complex light application conditions, which has the advantages of accurate pose estimation and solves the problem of inaccurate pose estimation of the traditional visual inertial navigation system algorithm in a complex lighting environment.

[0005] (2) Technical solutions

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions: a camera system adjustment method under complex light application conditions, comprising the following steps:

[0007] Step 1, establish an initialization module, an image acquisition module, an image preprocessing module, a pose optimization module, a data verification module, a data statistics module, a data calculation module and a management feedback module;

[0008] Step 2, the initialization module connects the camera through the network to calibrate the intrinsic and extrinsic parameters of the camera and set the initial exposure parameters;

[0009] Step 3, the image acquisition module acquires image data through the camera;

[0010] Step 4, the image preprocessing module converts the image data in the camera into standard format data and records it as original image data;

[0011] Step five, in the pose optimization module, a pose optimization simulation model is constructed, the model adjusts the original image data according to the simulation results, and the pose parameters are optimized, and the optimization data are recorded;

[0012] Step six, the optimization data of the pose optimization module are verified in the data verification module;

[0013] Step seven, the original image data and the optimization data are classified and numbered in the data statistics module;

[0014] Step eight, the numbered data of the data statistics module are calculated in the data calculation module;

[0015] Step nine, the calculation results are managed and feedback adjusted in the management feedback module.

[0016] Preferably, the original image data includes the brightness value of the original image, the actual z-th pose state value, the i-th feature point in the original image data, the i-th feature point of the original image re-projected under the pose X, and the total number of feature points in the original image data, the brightness value of the original image is numbered as Y1, Y2, Y3, … Y n , the actual z-th pose state value is numbered as H z , the i-th feature point in the original image data, the i-th feature point of the original image re-projected under the pose X, and the total number of feature points in the original image data are numbered as E i , Ev i (X), and m.

[0017] Preferably, the optimization data includes the model estimated illumination parameter corresponding to the brightness value of the original image, the z-th pose state value predicted by the simulation model, the j-th inertial measurement value in the optimization data, the j-th inertial measurement value predicted after the optimization image under the pose X, and the total number of inertial measurement values in the optimization data, the model estimated illumination parameter corresponding to the brightness value of the original image is numbered as Yc1, Yc2, Yc3, … Yc n , the z-th pose state value predicted by the simulation model is numbered as Ht z , the j-th inertial measurement value in the optimization data, the j-th inertial measurement value predicted after the optimization image under the pose X, and the total number of inertial measurement values in the optimization data are numbered as F j , Fv j (X), and k.

[0018] Preferably, the data computation module comprises a photometric correction unit, a pose estimation optimization unit and a simulation model evaluation unit, the photometric correction unit calculates a corrected image brightness value Lp according to the original image data and the optimization data, the pose estimation optimization unit calculates an optimal pose state value As according to the original image data and the optimization data, and the simulation model evaluation unit calculates a simulation model accuracy Rc according to the original image data and the optimization data.

[0019] Preferably, the photometric correction unit calculates a corrected image brightness value Lp according to the original image data and the optimization data, and the calculation formula is as follows:

[0020]

[0021] In the formula, Lp represents the corrected image brightness value, Y1, Y2, Y3, … Yn represent the brightness values of the original image, Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image, and Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image corresponding to the model estimated illumination parameters. n Y1, Y2, Y3, … Yn represent the brightness values of the original image, Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image, and Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image corresponding to the model estimated illumination parameters. i Y1, Y2, Y3, … Yn represent the brightness values of the original image, Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image, and Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image corresponding to the model estimated illumination parameters. n Y1, Y2, Y3, … Yn represent the brightness values of the original image, Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image, and Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image corresponding to the model estimated illumination parameters. i Y1, Y2, Y3, … Yn represent the brightness values of the original image, Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image, and Yc1, Yc2, Yc3, … Ycn represent the brightness values of the i-th original image corresponding to the model estimated illumination parameters.

[0022] Preferably, the corrected image brightness value Lp is used to adjust the exposure parameters of the camera.

[0023] Preferably, the pose estimation optimization unit calculates an optimal pose state value As according to the original image data and the optimization data, and the calculation formula is as follows:

[0024]

[0025] In the formula, As represents the optimal pose state value, Ei represents the i-th feature point in the original image data, Ev (X) represents the i-th feature point of the original image reprojected under the pose X, Fj represents the j-th inertial measurement value in the optimization data, Fv (X) represents the j-th inertial measurement value predicted under the pose X after the optimization image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values in the optimization data, α represents a weighting factor for balancing the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance for calculating the error. i Ei represents the i-th feature point in the original image data, Ev (X) represents the i-th feature point of the original image reprojected under the pose X, Fj represents the j-th inertial measurement value in the optimization data, Fv (X) represents the j-th inertial measurement value predicted under the pose X after the optimization image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values in the optimization data, α represents a weighting factor for balancing the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance for calculating the error. i Ev (X) represents the i-th feature point of the original image reprojected under the pose X, Fj represents the j-th inertial measurement value in the optimization data, Fv (X) represents the j-th inertial measurement value predicted under the pose X after the optimization image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values in the optimization data, α represents a weighting factor for balancing the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance for calculating the error. j Fj represents the j-th inertial measurement value in the optimization data, Fv (X) represents the j-th inertial measurement value predicted under the pose X after the optimization image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values in the optimization data, α represents a weighting factor for balancing the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance for calculating the error. j Fv (X) represents the j-th inertial measurement value predicted under the pose X after the optimization image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values in the optimization data, α represents a weighting factor for balancing the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance for calculating the error.

[0026] Preferably, the optimal pose state value As is used to guide the camera system to perform accurate pose estimation under complex lighting conditions.

[0027] Preferably, the simulation model evaluation unit calculates the simulation model accuracy Rc according to the original image data and the optimization data, and the calculation formula is:

[0028]

[0029] In the formula, Rc represents the simulation model accuracy, Ht z represents the zth pose state value predicted by the simulation model, H z represents the zth actual pose state value, N represents the total number of pose state values, and ‖*‖ represents the Euclidean distance for calculating the error.

[0030] Preferably, the simulation model accuracy Rc is used to evaluate the accuracy of the simulation model.

[0031] Compared with the prior art, the present application provides a camera system adjustment method under complex light application conditions, which has the following beneficial effects:

[0032] 1、The present application calculates the corrected image brightness value Lp according to the original image data and the optimization data through the luminosity correction unit, and this value is crucial for adjusting the exposure parameters of the camera, because the corrected image brightness value Lp can make the camera better adapt to the environmental changes under complex lighting conditions, so that by correcting the image brightness in the simulation model, the image distortion caused by uneven or sudden changes in lighting can be reduced, thereby improving the accuracy of subsequent pose estimation.

[0033] 2、The present application calculates the optimal pose state value As, combines visual and inertial information, and helps to provide an effective method for accurate pose estimation under complex lighting conditions. Under complex lighting conditions, visual information may be incomplete, by combining inertial information and establishing a simulation model, the information parameters of simulated vision can be summarized in the model to supplement the lack of visual information, thereby improving the accuracy of pose estimation. This method is suitable for dynamic environments where lighting conditions can change quickly, because inertial information can provide immediate feedback about camera motion, thereby helping the running system to quickly adapt to environmental changes. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 The flowchart of the method of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0036] Please refer to Figure 1 A camera system adjustment method under complex conditions of light application, comprising the following steps:

[0037] Step one, establish initialization module, image acquisition module, image preprocessing module, pose optimization module, data verification module, data statistics module, data calculation module and management feedback module;

[0038] Step two, the initialization module connects the camera through the network to calibrate the camera's internal and external parameters, and set the initial exposure parameters (ensure that the camera can work stably under different light environments);

[0039] Step three, the image acquisition module acquires image data through the camera;

[0040] Step four, the image preprocessing module converts the image data in the camera into standard format data and records as original image data;

[0041] Step five, in the pose optimization module, a pose optimization simulation model is constructed, the model adjusts the parameters of the original image data according to the simulation results, optimizes the pose parameters, and records the optimization data (records all related data in the optimization process to analyze and verify the optimization effect);

[0042] Step six, the optimization data of the pose optimization module is verified in the data verification module;

[0043] Step seven, the data statistics module classifies and numbers the original image data and the optimization data;

[0044] Step eight, the numbered data of the data statistics module is calculated in the data calculation module;

[0045] Step nine, the calculation results are managed and feedback adjusted in the management feedback module.

[0046] The original image data includes the brightness value of the original image, the actual z-th pose state value, the i-th feature point in the original image data, the i-th feature point of the original image reprojected under the pose X, and the total number of feature points in the original image data. The brightness value of the original image is numbered as Y1, Y2, Y3, … Y n , the actual z-th pose state value is numbered as H z , the i-th feature point in the original image data, the i-th feature point of the original image reprojected under the pose X, and the total number of feature points in the original image data are numbered as E i , Ev i (X), m.

[0047] The optimization data includes a model-estimated illumination parameter corresponding to a brightness value of the original image, a z-th pose state value predicted by the simulation model, a j-th inertial measurement value in the optimization data, a j-th inertial measurement value predicted after the optimization image under the pose X, and a total number of inertial measurement values in the optimization data, the model-estimated illumination parameter corresponding to the brightness value of the original image is numbered as Yc1, Yc2, Yc3, … Yc n , the z-th pose state value predicted by the simulation model is numbered as Ht z , the j-th inertial measurement value in the optimization data, the j-th inertial measurement value predicted after the optimization image under the pose X, and the total number of inertial measurement values in the optimization data are numbered as F j , Fv j (X), and k respectively.

[0048] The data calculation module includes a photometric correction unit, a pose estimation optimization unit, and a simulation model evaluation unit. The photometric correction unit calculates a corrected image brightness value Lp according to the original image data and the optimization data. The pose estimation optimization unit calculates an optimal pose state value As according to the original image data and the optimization data. The simulation model evaluation unit calculates a simulation model accuracy Rc according to the original image data and the optimization data.

[0049] The photometric correction unit calculates a corrected image brightness value Lp according to the original image data and the optimization data, and the calculation formula is as follows:

[0050]

[0051] In the formula, Lp represents the corrected image brightness value, Y1, Y2, Y3, … Y n represent the brightness values of the original image, Y i represents the brightness value of the i-th original image, Yc1, Yc2, Yc3, … Yc n represents the model-estimated illumination parameter corresponding to the brightness value of the original image, Yc i represents the model-estimated illumination parameter corresponding to the brightness value of the i-th original image, and n represents the number of statistical brightness values.

[0052] The advantage is that the photometric correction unit calculates a corrected image brightness value Lp according to the original image data and the optimization data, and this value is crucial for adjusting the exposure parameters of the camera. Because the corrected image brightness value Lp can make the camera better adapt to environmental changes under complex lighting conditions, it can reduce image distortion caused by uneven or sudden changes in illumination, thereby improving the accuracy of subsequent pose estimation.

[0053] The corrected image brightness value Lp is used to adjust the exposure parameters of the camera to adapt to environmental changes under complex lighting conditions.

[0054] The pose estimation optimization unit calculates an optimal pose state value As from the original image data and the optimization data, according to the following formula:

[0055]

[0056] In the formula, As represents the optimal pose state value, E i represents the i-th feature point in the original image data, Ev i (X) represents the i-th feature point of the original image re-projected under the pose X, F j represents the j-th inertial measurement value in the optimization data, Fv j (X) represents the j-th inertial measurement value predicted after the optimization image under the pose X, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values in the optimization data, a represents a weighting factor used to balance the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance used to calculate the error.

[0057] The advantage is that by calculating the optimal pose state value As, combining visual and inertial information, it helps to provide an effective method for accurate pose estimation under complex lighting conditions. Under complex lighting conditions, visual information may be incomplete, by combining inertial information and establishing a simulation model, the information parameters simulating visual information can be summarized in the model to supplement the lack of visual information, thereby improving the accuracy of pose estimation. This method is suitable for dynamic environments where lighting conditions can change quickly, because inertial information can provide immediate feedback about camera motion, helping the running system to quickly adapt to environmental changes.

[0058] The optimal pose state value As is used to guide the camera system to perform accurate pose estimation and adjustment under complex lighting conditions.

[0059] The simulation model evaluation unit calculates the simulation model accuracy Rc from the original image data and the optimization data, according to the following formula:

[0060]

[0061] In the formula, Rc represents the simulation model accuracy, Ht z represents the z-th pose state value predicted by the simulation model, H z represents the actual z-th pose state value, N represents the total number of pose state values, and ‖*‖ represents the Euclidean distance used to calculate the error.

[0062] The advantage is that the simulation model evaluation unit calculates the simulation model accuracy Rc according to the original image data and the optimization data, which can verify the performance of the simulation model in the data calculation module, and through the evaluation of the accuracy of the simulation model, problems existing in the algorithm of the simulation model can be found in time, and the problems can be improved in a targeted manner, thereby improving the application accuracy of the simulation model constructed by the application in a complex lighting environment.

[0063] The simulation model accuracy Rc is used to evaluate the accuracy of the simulation model, and the evaluation process is as follows:

[0064] S1, setting a threshold value: substituting historical data into the simulation model, and obtaining a preset accuracy threshold range by comparison;

[0065] S2, when the simulation model accuracy Rc is within the preset accuracy threshold range, it is considered that the accuracy of the simulation model meets the requirements, and the simulation model is effective;

[0066] S3, when the simulation model accuracy Rc is not within the preset accuracy threshold range, it is considered that the accuracy of the simulation model does not meet the requirements, and the simulation model needs to be adjusted again.

[0067] Although the embodiments of the application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the application, and the scope of the application is defined by the appended claims and their equivalents.

Claims

1. A method for adjusting a camera system under complex optical application conditions, characterized in that, Includes the following steps: Step 1: Establish the initialization module, image acquisition module, image preprocessing module, pose optimization module, data verification module, data statistics module, data calculation module, and management feedback module; Step 2: The initialization module connects to the camera via the network to calibrate the camera's intrinsic and extrinsic parameters, and to set the initial exposure parameters; Step 3: The image acquisition module acquires image data through the camera; Step 4: The image preprocessing module converts the image data from the camera into standard format data and records it as raw image data; Step 5: In the pose optimization module, construct a pose optimization simulation model. The model adjusts the parameters of the original image data based on the simulation results, optimizes the pose parameters, and records the optimization data. Step 6: Verify the optimized data from the pose optimization module in the data verification module; Step 7: The data statistics module classifies and numbers the raw image data and optimized data. Step 8: Calculate the numbered data from the data statistics module in the data calculation module; Step 9: The calculation results are managed in the management feedback module, and feedback adjustments are made accordingly. The data calculation module includes a photometric correction unit, a pose estimation optimization unit, and a simulation model evaluation unit. The photometric correction unit calculates the corrected image brightness value Lp based on the original image data and the optimized data. The pose estimation optimization unit calculates the optimal pose state value As based on the original image data and the optimized data. The simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and the optimized data. The pose estimation optimization unit calculates the optimal pose state value As based on the original image data and the optimized data. The calculation formula is as follows: In the formula, As represents the optimal pose state value, and E i Ev represents the i-th feature point in the original image data. i (X) represents the i-th feature point reprojected from the original image at pose X, F j Fv represents the j-th inertial measurement value in the optimized data. j (X) represents the j-th inertial measurement value predicted in pose X after image optimization, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values ​​in the optimized data, α represents the trade-off factor used to balance the importance of visual and inertial information, and ||*|| represents the Euclidean distance used to calculate the error.

2. The camera system adjustment method under complex optical application conditions according to claim 1, characterized in that: The original image data includes the brightness value of the original image, the actual z-th pose state value, the ith feature point in the original image data, the ith feature point of the original image reprojected under pose X, and the total number of feature points in the original image data. The brightness values ​​of the original image are numbered Y1, Y2, Y3, ... Y n The actual z-th pose state value is numbered H. z The i-th feature point in the original image data, the i-th feature point reprojected from the original image in pose X, and the total number of feature points in the original image data are respectively numbered E. i Ev i (X), m.

3. The camera system adjustment method under complex optical application conditions according to claim 1, characterized in that: The optimized data includes the illumination parameters estimated by the model corresponding to the brightness value of the original image, the z-th pose state value predicted by the simulation model, the j-th inertial measurement value in the optimized data, the j-th inertial measurement value predicted by the optimized image at pose X, and the total number of inertial measurement values ​​in the optimized data. The illumination parameters estimated by the model corresponding to the brightness value of the original image are numbered Yc1, Yc2, Yc3, ... Yc n The z-th pose state value predicted by the simulation model is numbered Ht. z The j-th inertial measurement value in the optimized data, the j-th inertial measurement value predicted in pose X after image optimization, and the total number of inertial measurement values ​​in the optimized data are respectively numbered F. j 、Fv j (X), k.

4. The camera system adjustment method under complex optical application conditions according to claim 1, characterized in that: The photometric correction unit calculates the corrected image brightness value Lp based on the original image data and the optimized data. The calculation formula is as follows: In the formula, Lp represents the corrected image brightness value, Y1, Y2, Y3, ... Y n Y represents the brightness value of the original image. i Let Yc1, Yc2, Yc3, ..., Yc represent the brightness values ​​of the original image at the i-th position. n Yc represents the illumination parameter estimated by the model corresponding to the brightness value of the original image. i This represents the illumination parameter estimated by the model corresponding to the brightness value of the i-th original image, and n represents the number of brightness values ​​counted.

5. The camera system adjustment method under complex optical application conditions according to claim 4, characterized in that: The corrected image brightness value Lp is used to adjust the camera's exposure parameters.

6. The camera system adjustment method under complex optical application conditions according to claim 1, characterized in that: The optimal pose state value As is used to guide the camera system to perform accurate pose estimation under complex lighting conditions.

7. The camera system adjustment method under complex optical application conditions according to claim 1, characterized in that: The simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and optimized data, and the calculation formula is as follows: In the formula, Rc represents the accuracy of the simulation model, and Ht z H represents the z-th pose state value predicted by the simulation model. z This represents the actual z-th pose state value, N represents the total number of pose state values, and ||*|| represents the Euclidean distance used to calculate the error.

8. The camera system adjustment method under complex optical application conditions according to claim 7, characterized in that: The accuracy Rc of the simulation model is used to evaluate the accuracy of the simulation model.

Images