Camera system adjusting method under complex light application condition

Abstract

The invention relates to the technical field of imaging, and discloses a camera system adjusting method under a light application complex condition, and the method comprises the steps: building an initialization module, an image collection module, an image preprocessing module, a pose optimization module, a data verification module, a data statistics module, a data calculation module and a management feedback module, the image acquisition module acquires image data, the image preprocessing module is used for preprocessing the data, the pose optimization module constructs a pose optimization simulation model, the model performs parameter adjustment on original image data according to a simulation result, optimizes pose parameters, records optimized data, and transmits the optimized data to the image processing module. The data verification module verifies the optimized data of the pose optimization module, the data statistics module performs data classification and numbering on the original image data and the optimized data, the data calculation module calculates the numbered data of the data statistics module, the calculation result is managed in the management feedback module, and feedback adjustment is performed.

Description

Technical Field

[0001] The present invention relates to the field of imaging technology, and in particular to a camera system adjustment method under complex light application conditions. Background Art

[0002] In complex lighting environments, the pose estimation of traditional visual inertial navigation system (VINS) algorithms may be inaccurate. This is because factors such as lighting and camera exposure will affect the grayscale of the image, thereby affecting the accuracy of visual information, resulting in a decrease in the quality of pose estimation. This problem is particularly prominent under complex light application conditions, because under these conditions, the environmental perception and adaptive adjustment capabilities of the imaging device are limited, and the analysis of ambient light changes is relatively "rough", and a comprehensive quantitative description cannot be made. In addition, in order to fully utilize the advantages of the high frame rate of IMU information and meet the real-time frame rate requirements of the image stream, mainstream visual inertial SLAM methods mostly use semi-direct methods to deal with image data association problems. Although this improves real-time performance, it may sacrifice accuracy in some cases. Therefore, a camera system adjustment method that can improve the accuracy of pose estimation under complex lighting conditions is needed. Summary of the invention

[0003] 1. Technical issues to be solved

[0004] In view of the shortcomings of the prior art, the present invention provides a camera system adjustment method under complex light application conditions, which has the advantage of accurately estimating the posture and posture, and solves the problem of inaccurate posture estimation of traditional visual inertial navigation system algorithms in complex lighting environments.

[0005] (II) Technical solution

[0006] To achieve the above object, the present invention provides the following technical solution: a camera system adjustment method under complex light application conditions, comprising the following steps:

[0007] Step 1: Establish an initialization module, an image acquisition module, an image preprocessing module, a posture optimization module, a data verification module, a data statistics module, a data calculation module, and a management feedback module;

[0008] Step 2: The initialization module connects to the camera through the network to calibrate the camera's intrinsic and extrinsic parameters, and set the initial exposure parameters;

[0009] Step 3: The image acquisition module acquires image data through a camera;

[0010] Step 4: The image preprocessing module converts the image data in the camera into standard format data and records it as raw image data;

[0011] Step 5: In the posture optimization module, a posture optimization simulation model is constructed. The model adjusts the parameters of the original image data according to the simulation results, optimizes the posture parameters, and records the optimization data;

[0012] Step 6: Verify the optimized data of the posture optimization module in the data verification module;

[0013] Step 7: The data statistics module classifies and numbers the original image data and the optimized data;

[0014] Step 8: Calculate the numbered data of the data statistics module in the data calculation module;

[0015] Step 9. The calculation results are managed in the management feedback module, and feedback adjustments are made.

[0016] Preferably, the original image data includes the brightness value of the original image, the actual z-th posture state value, the i-th feature point in the original image data, the i-th feature point reprojected from the original image at posture x, and the total number of feature points in the original image data, and the brightness values ​​of the original image are numbered as Y1, Y2, Y3, ...Y n , the actual z-th posture state value is numbered as H z The i-th feature point in the original image data, the i-th feature point reprojected from the original image at position X, and the total number of feature points in the original image data are numbered E i , Ev i (X), m.

[0017] Preferably, the optimization data includes the illumination parameters estimated by the model corresponding to the brightness value of the original image, the zth posture state value predicted by the simulation model, the jth inertial measurement value in the optimization data, the jth inertial measurement value predicted at posture X after the image is optimized, and the total number of inertial measurement values ​​in the optimization data. The illumination parameters estimated by the model corresponding to the brightness value of the original image are numbered as Yc1, Yc2, Yc3, ...Yc n The zth posture state value predicted by the simulation model is numbered as Ht z The jth inertial measurement value in the optimized data, the jth inertial measurement value predicted at the pose X after optimizing the image, and the total number of inertial measurement values ​​in the optimized data are numbered F j 、Fv j (X), k.

[0018] Preferably, the data calculation module includes a photometric correction unit, a posture estimation optimization unit and a simulation model evaluation unit, the photometric correction unit calculates the corrected image brightness value Lp based on the original image data and the optimized data, the posture estimation optimization unit calculates the optimal posture state value As based on the original image data and the optimized data, and the simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and the optimized data.

[0019] Preferably, the photometric correction unit calculates the corrected image brightness value Lp according to the original image data and the optimized data, and the calculation formula is:

[0020]

[0021] In the formula, Lp represents the corrected image brightness value, Y1, Y2, Y3, ...Y n Represents the brightness value of the original image, Y i Represents the brightness value of the i-th original image, Yc1, Yc2, Yc3, ...Yc n Indicates the illumination parameter estimated by the model corresponding to the brightness value of the original image, Yc i represents the illumination parameter estimated by the model corresponding to the brightness value of the i-th original image, and n represents the number of brightness values ​​counted.

[0022] Preferably, the corrected image brightness value Lp is used to adjust the exposure parameters of the camera.

[0023] Preferably, the posture estimation optimization unit calculates the optimal posture state value As according to the original image data and the optimized data, and the calculation formula is:

[0024]

[0025] In the formula, As represents the optimal posture state value, E i Represents the i-th feature point in the original image data, Ev i (X) represents the i-th feature point of the original image reprojected at position X, F j represents the jth inertial measurement value in the optimization data, Fv j (X) represents the jth inertial measurement value predicted at pose X after optimizing the image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values ​​in the optimized data, α represents the trade-off factor used to balance the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance, which is used to calculate the error.

[0026] Preferably, the optimal posture state value As is used to guide the camera system to perform accurate posture estimation under complex lighting conditions.

[0027] Preferably, the simulation model evaluation unit calculates the simulation model accuracy Rc according to the original image data and the optimized data, and the calculation formula is:

[0028]

[0029] In the formula, Rc represents the accuracy of the simulation model, Ht z represents the z-th posture state value predicted by the simulation model, H z represents the actual z-th pose state value, N represents the total number of pose state values, and ‖*‖ represents the Euclidean distance, which is used to calculate the error.

[0030] Preferably, the simulation model accuracy Rc is used to evaluate the accuracy of the simulation model.

[0031] Compared with the prior art, the present invention provides a camera system adjustment method under complex light application conditions, which has the following beneficial effects:

[0032] 1. The present invention calculates the corrected image brightness value Lp according to the original image data and the optimized data through the photometric correction unit. It is very important that this value can be used to adjust the exposure parameters of the camera, because the corrected image brightness value Lp can enable the camera to better adapt to environmental changes under complex lighting conditions. Therefore, by correcting the image brightness in the simulation model, the image distortion caused by uneven or sudden lighting can be reduced, thereby improving the accuracy of subsequent pose estimation.

[0033] 2. The present invention calculates the optimal pose state value As and combines visual and inertial information to help provide an effective method for accurate pose estimation under complex lighting conditions. Under complex lighting conditions, visual information may be incomplete. By combining inertial information and establishing a simulation model, the information parameters of simulated vision are summarized in the model to supplement the deficiency of visual information, thereby improving the accuracy of pose estimation. This method is suitable for dynamic environments where lighting conditions can change rapidly, because inertial information can provide instant feedback on camera movement, thereby helping the operating system to quickly adapt to environmental changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See also Figure 1 , a camera system adjustment method under complex light application conditions, comprising the following steps:

[0037] Step 1: Establish an initialization module, an image acquisition module, an image preprocessing module, a posture optimization module, a data verification module, a data statistics module, a data calculation module, and a management feedback module;

[0038] Step 2: The initialization module connects to the camera through the network to calibrate the camera's intrinsic and extrinsic parameters, and set the initial exposure parameters (to ensure that the camera can work stably under different lighting environments);

[0039] Step 3: The image acquisition module acquires image data through a camera;

[0040] Step 4: The image preprocessing module converts the image data in the camera into standard format data and records it as raw image data;

[0041] Step 5: In the posture optimization module, a posture optimization simulation model is constructed. The model adjusts the parameters of the original image data according to the simulation results, optimizes the posture parameters, and records the optimization data (records all relevant data during the optimization process to analyze and verify the optimization effect);

[0042] Step 6: Verify the optimized data of the posture optimization module in the data verification module;

[0043] Step 7: The data statistics module classifies and numbers the original image data and the optimized data;

[0044] Step 8: Calculate the numbered data of the data statistics module in the data calculation module;

[0045] Step 9. The calculation results are managed in the management feedback module, and feedback adjustments are made.

[0046] The original image data includes the brightness value of the original image, the actual z-th posture state value, the i-th feature point in the original image data, the i-th feature point reprojected from the original image at posture x, and the total number of feature points in the original image data. The brightness values ​​of the original image are numbered as Y1, Y2, Y3, ...Y n , the actual z-th posture state value is numbered H z The i-th feature point in the original image data, the i-th feature point reprojected from the original image at pose X, and the total number of feature points in the original image data are numbered E i , Ev i (X), m.

[0047] The optimized data includes the illumination parameters estimated by the model corresponding to the brightness value of the original image, the zth posture state value predicted by the simulation model, the jth inertial measurement value in the optimized data, the jth inertial measurement value predicted at posture X after the optimized image, and the total number of inertial measurement values ​​in the optimized data. The illumination parameters estimated by the model corresponding to the brightness value of the original image are numbered as Yc1, Yc2, Yc3, ...Yc n , the zth posture state value predicted by the simulation model is numbered as Ht z The jth inertial measurement value in the optimized data, the jth inertial measurement value predicted at pose X after optimizing the image, and the total number of inertial measurement values ​​in the optimized data are numbered F j 、Fv j (X), k.

[0048] The data calculation module includes a photometric correction unit, a posture estimation and optimization unit and a simulation model evaluation unit. The photometric correction unit calculates the corrected image brightness value Lp based on the original image data and the optimized data. The posture estimation and optimization unit calculates the optimal posture state value As based on the original image data and the optimized data. The simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and the optimized data.

[0049] The photometric correction unit calculates the corrected image brightness value Lp according to the original image data and the optimized data, and the calculation formula is:

[0050]

[0051] In the formula, Lp represents the corrected image brightness value, Y1, Y2, Y3, ...Y n Represents the brightness value of the original image, Y i Represents the brightness value of the i-th original image, Yc1, Yc2, Yc3, ...Yc n Indicates the illumination parameter estimated by the model corresponding to the brightness value of the original image, Yc i represents the illumination parameter estimated by the model corresponding to the brightness value of the i-th original image, and n represents the number of brightness values ​​counted.

[0052] The advantages are: the photometric correction unit calculates the corrected image brightness value Lp based on the original image data and the optimized data. This value can be used to adjust the exposure parameters of the camera, which is crucial because the corrected image brightness value Lp can enable the camera to better adapt to environmental changes under complex lighting conditions. By correcting the image brightness in the simulation model, the image distortion caused by uneven or sudden lighting can be reduced, thereby improving the accuracy of subsequent pose estimation.

[0053] The corrected image brightness value Lp is used to adjust the camera's exposure parameters to adapt to environmental changes under complex lighting conditions.

[0054] The pose estimation optimization unit calculates the optimal pose state value As based on the original image data and the optimized data, and its calculation formula is:

[0055]

[0056] In the formula, As represents the optimal posture state value, E i Represents the i-th feature point in the original image data, Ev i (X) represents the i-th feature point of the original image reprojected at position X, F j represents the jth inertial measurement value in the optimization data, Fv j (X) represents the jth inertial measurement value predicted at pose X after optimizing the image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values ​​in the optimized data, α represents the trade-off factor used to balance the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance, which is used to calculate the error.

[0057] The advantages are: by calculating the optimal pose state value As and combining visual and inertial information, it helps to provide an effective method for accurate pose estimation under complex lighting conditions. Under complex lighting conditions, visual information may be incomplete. By combining inertial information and establishing a simulation model, the information parameters of simulated vision are summarized in the model to supplement the deficiency of visual information, thereby improving the accuracy of pose estimation. This method is suitable for dynamic environments where lighting conditions can change rapidly, because inertial information can provide instant feedback about camera movement, thereby helping the operating system to quickly adapt to environmental changes.

[0058] The optimal pose state value As is used to guide the camera system to perform accurate pose estimation and adjustment under complex lighting conditions.

[0059] The simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and the optimized data, and the calculation formula is:

[0060]

[0061] In the formula, Rc represents the accuracy of the simulation model, Ht z represents the z-th posture state value predicted by the simulation model, H z represents the actual z-th pose state value, N represents the total number of pose state values, and ‖*‖ represents the Euclidean distance, which is used to calculate the error.

[0062] The advantages are: the simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and the optimized data. This step can verify the performance of the simulation model in the data calculation module. By evaluating the accuracy of the simulation model, problems in the algorithm of the simulation model can be discovered in time, and targeted improvements can be made to the problems, thereby improving the application accuracy of the simulation model constructed by the present invention in complex lighting environments.

[0063] The simulation model accuracy Rc is used to evaluate the accuracy of the simulation model, and its evaluation process is:

[0064] S1. Setting threshold: Substitute historical data into the simulation model and obtain the preset accuracy threshold range through comparison;

[0065] S2. When the simulation model accuracy Rc is within the preset accuracy threshold range, it is considered that the accuracy of the simulation model meets the requirements and the simulation model is effective;

[0066] S3. When the simulation model accuracy Rc is not within the preset accuracy threshold range, it is considered that the accuracy of the simulation model does not meet the requirements and the simulation model needs to be readjusted.

[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A camera system adjustment method under complex light application conditions, characterized in that: The following steps are involved: Step 1: Establish an initialization module, an image acquisition module, an image preprocessing module, a posture optimization module, a data verification module, a data statistics module, a data calculation module, and a management feedback module; Step 2: The initialization module connects to the camera through the network to calibrate the camera's intrinsic and extrinsic parameters, and set the initial exposure parameters; Step 3: The image acquisition module acquires image data through a camera; Step 4: The image preprocessing module converts the image data in the camera into standard format data and records it as raw image data; Step 5: In the posture optimization module, a posture optimization simulation model is constructed. The model adjusts the parameters of the original image data according to the simulation results, optimizes the posture parameters, and records the optimization data; Step 6: Verify the optimized data of the posture optimization module in the data verification module; Step 7: The data statistics module classifies and numbers the original image data and the optimized data; Step 8: Calculate the numbered data of the data statistics module in the data calculation module; Step 9. The calculation results are managed in the management feedback module, and feedback adjustments are made.

2. The camera system adjustment method under complex light application conditions according to claim 1, characterized in that: The original image data includes the brightness value of the original image, the actual z-th posture state value, the i-th feature point in the original image data, the i-th feature point reprojected from the original image at posture x, and the total number of feature points in the original image data. The brightness values ​​of the original image are numbered as Y1, Y2, Y3, ...Y n , the actual z-th posture state value is numbered as H z The i-th feature point in the original image data, the i-th feature point reprojected from the original image at position X, and the total number of feature points in the original image data are numbered E i , Ev i (X), m.

3. The camera system adjustment method under complex light application conditions according to claim 1, characterized in that: The optimization data includes the illumination parameters estimated by the model corresponding to the brightness value of the original image, the zth posture state value predicted by the simulation model, the jth inertial measurement value in the optimization data, the jth inertial measurement value predicted at posture X after the image is optimized, and the total number of inertial measurement values ​​in the optimization data. The illumination parameters estimated by the model corresponding to the brightness value of the original image are numbered as Yc1, Yc2, Yc3, ...Yc n The zth posture state value predicted by the simulation model is numbered as Ht z The jth inertial measurement value in the optimized data, the jth inertial measurement value predicted at the pose X after optimizing the image, and the total number of inertial measurement values ​​in the optimized data are numbered F j 、Fv j (X), k.

4. The camera system adjustment method under complex light application conditions according to claim 3, characterized in that: The data calculation module includes a photometric correction unit, a posture estimation optimization unit and a simulation model evaluation unit. The photometric correction unit calculates the corrected image brightness value Lp based on the original image data and the optimized data. The posture estimation optimization unit calculates the optimal posture state value As based on the original image data and the optimized data. The simulation model evaluation unit calculates the simulation model accuracy Rc based on the original image data and the optimized data.

5. The camera system adjustment method under complex light application conditions according to claim 4, characterized in that: The photometric correction unit calculates the corrected image brightness value Lp according to the original image data and the optimized data, and the calculation formula is: In the formula, Lp represents the corrected image brightness value, Y1, Y2, Y3, ...Y n Represents the brightness value of the original image, Y i Represents the brightness value of the i-th original image, Yc1, Yc2, Yc3, ...Yc n Indicates the illumination parameter estimated by the model corresponding to the brightness value of the original image, Yc i represents the illumination parameter estimated by the model corresponding to the brightness value of the i-th original image, and n represents the number of brightness values ​​counted.

6. The camera system adjustment method under complex light application conditions according to claim 5, characterized in that: The corrected image brightness value Lp is used to adjust the exposure parameters of the camera.

7. The camera system adjustment method under complex light application conditions according to claim 4, characterized in that: The posture estimation optimization unit calculates the optimal posture state value As according to the original image data and the optimized data, and the calculation formula is: In the formula, As represents the optimal posture state value, E i Represents the i-th feature point in the original image data, Ev i (X) represents the i-th feature point of the original image reprojected at pose X, F j represents the jth inertial measurement value in the optimization data, Fv j (X) represents the jth inertial measurement value predicted at pose X after optimizing the image, m represents the total number of feature points in the original image data, k represents the total number of inertial measurement values ​​in the optimized data, α represents the trade-off factor used to balance the importance of visual information and inertial information, and ‖*‖ represents the Euclidean distance, which is used to calculate the error.

8. The camera system adjustment method under complex light application conditions according to claim 7, characterized in that: The optimal pose state value As is used to guide the camera system to perform accurate pose estimation under complex lighting conditions.

9. The camera system adjustment method under complex light application conditions according to claim 4, characterized in that: The simulation model evaluation unit calculates the simulation model accuracy Rc according to the original image data and the optimized data, and the calculation formula is: In the formula, Rc represents the accuracy of the simulation model, Ht z represents the z-th posture state value predicted by the simulation model, H z represents the actual z-th pose state value, N represents the total number of pose state values, and ‖*‖ represents the Euclidean distance, which is used to calculate the error.

10. The camera system adjustment method under complex light application conditions according to claim 9, characterized in that: The simulation model accuracy Rc is used to evaluate the accuracy of the simulation model.

Images