Fully automatic camera calibration system and method for adaptively adjusting brightness and blur

Abstract

The present invention relates to the field of computer vision technology, in particular to a fully automatic camera calibration system and method for adaptively adjusting brightness and blur degree. The system includes a monocular camera, a rotary moving platform, a platform motion controller, a center calibration board, a wireless transmission device, a processor, a display device, and an automatic high-precision calibration software; the method converts the proportion of the number of pixels with black and white gray values in a specific area of the image into 1:2 according to a compensation coefficient, and adaptively adjusts the exposure value according to the average gray value of the area; by extracting the edge points in four directions of a specific feature circle in the image, fitting a curve to the gray values of the edge points and calculating the average slope of the edge points, the value of the average slope characterizes the current blur situation of the image, which is fed back to the platform controller to realize adaptive adjustment of the blur degree, controlling the rotary moving platform to adjust to a preset optimal pose to automatically capture an image according to a preset instruction, and realizing automatic high-precision calibration of the camera through a high-precision camera calibration strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of camera computer vision, in particular to a fully automatic camera calibration system and method for adaptively adjusting brightness and blur degree. Background Art

[0002] In modern highly developed automated systems, machine vision positioning systems are widely used in various fields, such as robot vision guidance, three-dimensional on-line dimensional measurement, etc. At present, there is a strong demand for high-precision machine vision positioning systems in the automation field. The premise of improving the positioning accuracy of machine vision is to improve the calibration accuracy of industrial cameras. However, the existing traditional industrial camera calibration technologies are mainly divided into self-calibration methods, active calibration methods, and traditional calibration methods. Among them, the traditional calibration method is widely used in industry because of its simple principle and high calibration accuracy. It is mainly divided into Zhang Zhengyou calibration method and two-step method. Among them, the Zhang Zhengyou calibration method only requires a calibration board to achieve camera calibration, so it is the most commonly used. At present, great progress has been made in camera calibration algorithms. K. Sirisantisamrid et al. used the iterative linear method to calculate the lens distortion coefficient, principal point, and distance between cameras to determine the parameters of the camera; Wang Shoukun et al. used the sub-pixel edge detection algorithm to extract the edges of the feature circle, and proposed a Zhang's camera calibration method based on a circular array calibration board by imposing conditional restrictions on the edge closed features; Peng Yan, Guo Junbin et al. considered the perspective deviation and proposed a high-precision camera calibration method based on plane transformation; Huang Wanting et al. used MATLAB for image preprocessing, and improved the accuracy of corner detection through methods such as harris corner detection and sub-pixel refinement to achieve high-precision camera calibration. The above methods mainly use manual placement of the marker board for calibration calculation. The calibration process is cumbersome and time-consuming. The azimuth angle of the calibration board is not easy to judge, the calibration angles are not unified, and there are also problems such as the brightness and clarity of the marker board cannot be judged, and the calibration accuracy is easily affected by human factors. Therefore, automatically moving the marker to the optimal working distance, using the optimal calibration angle, and obtaining high-quality images to complete the efficient and high-precision calibration of the camera are crucial for the realization of the visual high-precision positioning system.

[0003] The existing fully automatic calibration system can conveniently and efficiently complete the calculation of the internal parameters of the camera, and at the same time can overcome the problem of insufficient calculation accuracy caused by manual operation. Wang Jianfeng et al. from Chang'an University drive the marker through a conveyor belt, and the monocular camera continuously takes pictures to achieve automatic camera calibration; Xu Yin et al. design a robotic arm to control the movement of the marker board, and the acquisition device is fixed on the robotic arm and takes pictures of the calibration board to achieve camera calibration; Xun Yi et al. from Zhejiang University design a three-degree-of-freedom motion platform to support the checkerboard target paper, input the expected pitch attitude, yaw attitude and roll attitude through the PC host computer, and the camera collects the images of the calibration board in different postures to achieve automatic camera calibration; currently, the systems and methods of fully automatic calibration do not judge and adaptively adjust the image blurriness and brightness, and rely on manual operation or random shooting in the pose of the marker, resulting in a great impact on the camera calibration accuracy and low robustness. Summary of the Invention

[0004] The purpose of the present invention is to solve the defects of the existing technology. The present application proposes a fully automatic camera calibration system and method for adaptively adjusting brightness and blurriness. This system can adaptively adjust the image blurriness, thereby determining the optimal working distance of the industrial camera and dynamically adjusting the exposure value. At the same time, by automatically adjusting the optimal shooting angle, images of the calibration board from different perspectives are obtained, and finally high-precision camera parameters are calculated.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is:

[0006] The fully automatic camera calibration system for adaptively adjusting brightness and blurriness mainly consists of a monocular camera, a rotary moving platform, a platform motion controller, a center calibration board, a wireless transmission device, a processor, a display device and an automatic high-precision calibration software; this calibration system includes: the monocular camera is fixed at a certain position, and the corresponding rotary moving platform is below the camera. The initial value of the distance between the rotary moving platform and the camera is calculated through the working distance. The platform motion controller drives the calibration board supported by the rotary moving platform to perform pitch, yaw and roll and adjust the distance between the calibration board and the camera. The monocular camera takes pictures of the center calibration board in different poses to complete camera calibration;

[0007] Below the monocular camera are a rotating mobile platform and a platform motion controller. The rotating mobile platform supports the center calibration board. The initial value of the distance between the rotating mobile platform and the camera is calculated through the working distance. The monocular camera collects the current image of the center calibration board and transmits it back to the automatic high-precision calibration software, which displays the current image on the display device. The automatic high-precision calibration software adjusts parameters such as the camera gain and acquisition mode online to improve the image quality, extracts the rectangular area of the adjusted image, and converts the size ratio of the black-and-white area to 1:2. The exposure value is adaptively adjusted through the average gray level of the rectangular area. The image with appropriate brightness is subjected to circle detection by the EDcircle algorithm, and the edge gray points in four directions of the feature circle are extracted. The gray points are fitted to a curve, and the average slope of the gray points is calculated to characterize the blurriness of the image, and the result is fed back to the platform controller to control the rotating mobile platform to adjust the distance in the Z direction to achieve adaptive adjustment of the blurriness;

[0008] The platform motion controller controls the rotating mobile platform to adjust to the preset optimal pose group through preset commands. The poses achieved include pitch pose, yaw pose, roll pose, and the distance from the camera. The rotating mobile platform supports the movement of the calibration board to achieve high-precision calibration of the camera;

[0009] A further improvement of the technical solution of the present invention lies in: the monocular camera, which consists of a camera and a lens, is used to collect the images of the calibration board switched by the rotating mobile platform. The initial position of the camera is ensured to be directly above the rotating mobile platform;

[0010] The rotating mobile platform is a four-degree-of-freedom mechanical structure. Each degree of freedom is realized by a motor to adjust the angle. The motors are uniformly controlled by the platform motion controller, which can realize the angle switching and adjustment of the pitch attitude, yaw attitude, and roll attitude and the distance from the camera, and is mainly used to adjust the pose of the center calibration board;

[0011] The platform motion controller takes the stm32 development board as the core, is connected to the motors on the rotating mobile platform by connecting wires, and the other end of the stm32 is connected to the receiving end of the wireless transmission module. It is mainly used to control the rotating mobile platform to switch poses. Through the preset optimal calibration board pose positions, the operator sends a start calibration command through the high-precision camera calibration software. The platform motion controller controls the rotating mobile platform to rotate to the preset positions one by one. The camera automatically recognizes the poses according to the switching situation of the rotating mobile platform and saves the images for calibration calculation;

[0012] The center calibration board is an image used for camera calibration calculation. Through different poses of the center calibration board, the three-dimensional coordinates of the world coordinate system and the two-dimensional coordinates of the image coordinate system of the calibration board are extracted to achieve camera calibration;

[0013] A wireless transmission device is used to send commands to the motion controller of the control platform and send corresponding commands to the motion controller through the wireless transmission device according to the operation of the high-precision camera calibration software;

[0014] A processor is used to implement data acquisition, storage, and internal parameter calibration methods, providing a carrier for the internal parameter calibration software;

[0015] A display device is used for operators to operate the high-precision camera calibration software and to display the image information collected by the camera in real time;

[0016] The automatic calibration software includes a function setting area, a function view area, a display view area, and a high-precision calibration algorithm; among them, the function setting area provides two functions required for camera calibration, namely camera parameter setting and camera calibration, and operators can select the functions according to the actual situation; the function view area specifically displays the selected calibration function; the display view area is used to display the images obtained by the camera.

[0017] A fully automatic camera calibration method for adaptively adjusting brightness and blur includes the following steps:

[0018] Step 1: Calculate the initial position of the rotating and moving platform. According to the pinhole imaging model, the camera has a certain working distance range. Since the current focal length of the camera is unknown, the focal length f is estimated, and the initial position WD of the rotating and moving platform can be solved through the distance formula;

[0019] Step 2: Set the camera parameters. Switch to the camera parameter interface through the function setting area, obtain the camera model and the current image, set the working state of the camera, set the camera acquisition mode and automatic white balance, and manually coarsely adjust the exposure value and camera gain according to the on-site environment, so as to improve the quality of the obtained images;

[0020] Step 3: Adaptively adjust the exposure. The color composition of the circular calibration board is black and white. The calibration board is directly below the lens. Count the proportion R of the number of black and white pixels in a fixed-size rectangular area at the center of the image n , and according to the proportion R of the number of black and white pixels with moderate brightness o for the proportion R within the currently counted fixed area n perform conversion of the proportion of the number of black and white pixels and calculate the average gray value of the current area, and adaptively adjust the exposure according to the average gray value;

[0021] Step 4: Adaptive adjustment of blurriness. By sorting the feature circles on the center calibration plate, obtain the gray values of the four azimuth edges of multiple specific feature circles and perform curve fitting separately. Calculate the average slope of the gray values in each direction, compare the average slopes in the same direction among multiple circles, and remove the maximum value. Calculate the average slope of the edge points in multiple directions of multiple circles as the evaluation factor for the image blurriness, and achieve adaptive adjustment of blurriness according to the evaluation factor.

[0022] Step 5: Obtain the optimal calibration angle. Preset a set of optimal calibration angles on the rotating moving platform and record them in the platform motion controller. The high-precision camera calibration software sends a start calibration instruction to the platform motion controller through a wireless transmission device. The rotating moving platform rotates the preset poses one by one, and the camera captures 18 stable and clear calibration images at different pose distances.

[0023] Step 6: Camera calibration. For each of the 18 calibration images, obtain the center of the feature circle and calculate the camera internal parameters and distortion coefficients according to Zhang Zhengyou calibration method. The data results are generated into a calibration log and saved in the database for the operator to view and use.

[0024] A further improvement of the technical solution of the present invention lies in that in step 3, the adaptive adjustment of exposure realizes its function through the following steps:

[0025] Step 3.1: Calculate the average gray value within the rectangular area. Select a rectangular area with a certain area within the center calibration plate area, set the quantity ratio of standard black and white as R o , calculate the current quantity ratio R n of black and white and obtain the current area black and white compensation parameters B o and W c based on R c . Convert the current quantity ratio of black and white into the same as the set standard value through the compensation parameters, and thus obtain the average gray value after adding the compensation parameters. Among them, the expressions of compensation parameters B c and W c are:

[0026]

[0027]

[0028] where, b and b n represent the values of the standard ratio and the number of black pixels in the nth specific area, and w and w n represent the values of the standard ratio and the number of white pixels in the nth specific area;

[0029] The expression formula of the average gray value G avec is:

[0030]

[0031] Among them, A b represents the weight coefficient of the black area, and A w represents the weight coefficient of the white area. Since the gray value of the black area is much smaller than that of the white area, increasing the weight coefficient A b of the black area results in a more significant change in the average gray value when the brightness changes;

[0032] Step 3.2, adaptively adjust the exposure: Collect a series of brightness images from under-exposure to over-exposure and calculate the average gray value G avec respectively. Construct the brightness distribution of the average gray value based on compensation and weight distribution, calculate the average gray value of the currently selected area, and deduce the brightness adjustment strategy based on the brightness distribution to achieve adaptive adjustment of the exposure; The brightness adjustment strategy is as follows:

[0033]

[0034] Among them, E represents the modified exposure value, L represents the brightness situation, and K L represents the exposure adjustment coefficient. The average gray value G avec of the currently obtained area, G up and G under represent the average gray values of the upper and lower limits of suitable brightness within the rectangular area; By correcting the value of E, the brightness can be adaptively adjusted to obtain a clear calibration image.

[0035] A further improvement of the technical solution of the present invention lies in: In step 4, the adaptive adjustment of the blur degree is realized through the following steps:

[0036] Step 4.1, feature circle sorting: Detect the feature circles in the image, remove the interfering feature circles and approximate circles, sort the feature circles, and select multiple specific feature circles as the processing objects;

[0037] Step 4.2, edge point evaluation index: Count the edge points of the selected multiple feature circles in four directions, deduce the edge point evaluation index based on the different gray value change characteristics of the edge points and the internal and external pixel points of the feature circles, and filter out the edge pixel points based on this; The expression of the edge point evaluation index is:

[0038]

[0039] Among them, M n represents whether the nth point is an edge point evaluation index, G n represents the gray value of the nth point, and d represents the step size;

[0040] Step 4.3, Ambiguity evaluation index: Based on the pixel values of the edge points in four directions of multiple feature circles, perform polynomial curve fitting, calculate the slope of each edge point pixel value, and screen out the average slope with a large difference from the average slope of the edge point pixel values of other feature circles in the same direction, so as to calculate the average slope of the edge point pixel values in multiple directions of multiple circles as the evaluation index of image ambiguity; The expression of the ambiguity evaluation index is as follows:

[0041]

[0042] where m is the number of edge points, Y is the expression of the edge point pixel values after polynomial fitting, z represents the direction of the edge point, and i represents the number of feature circles obtained;

[0043] Step 4.4, Adaptive adjustment of ambiguity: Judge the current image quality according to the ambiguity evaluation factor, and adjust the distance between the camera and the center calibration plate through the adaptive image ambiguity adjustment strategy to improve the image quality; The expression of adaptive ambiguity adjustment is as follows:

[0044]

[0045] where D n represents the direction of the camera moving distance, A k and K o represent fixed coefficients and the image clarity threshold, t n represents the direction weight. In the adaptive ambiguity adjustment of the distance between the camera and the marker, t n represents the judgment of the change in ambiguity after the current movement, and t n is determined according to the following relationship:

[0046] t n = K n - K n-1

[0047] where K n represents the ambiguity evaluation factor of the nth image.

[0048] A further improvement of the technical solution of the present invention is that in step 5, the following steps are included:

[0049] Step 5.1, preset the optimal calibration angle: If the inclination angles of the calibration board in pitch, yaw, and roll are too large, the feature circles will be distorted. Through repeated experiments, the optimal calibration angle is obtained and the preset optimal calibration angle is recorded in the platform motion controller. The number of optimal calibration angles is set to 18, which are collected in three groups, and each group has six fixed poses: pitch 5° roll -5°, pitch 5° roll -10°, pitch -5° roll -15°, pitch 5° roll -20°, pitch 10° roll -15°, pitch 10° roll -20°; The three groups of calibration images are divided as follows: The distance between the first group of cameras and the pose rotating table is WD + D n , the distance between the second group of cameras and the pose rotating table is WD + D n + WD1, and the distance between the third group of cameras and the pose rotating table is WD + D n + WD1 and the pose rotating table is yawed 5°, where D n represents the distance moved for adaptive adjustment of the blur, and WD1 represents one-fifth of the depth of field of the current lens;

[0050] Step 5.2, calibration image acquisition: The automatic high-precision calibration software sends a start calibration command to the platform controller through a wireless transmission device and controls the rotating and moving platform to rotate the preset angles in sequence. The automatic high-precision calibration software automatically identifies the state of the rotating and moving platform and saves the stable and clear calibration images; The high-precision camera calibration system can adaptively adjust the exposure and adaptively adjust the blur to improve the quality of the acquired images. By moving the rotating and moving platform to the preset optimal calibration angles in sequence, data such as the camera internal parameters and distortion coefficients are calculated based on the acquired calibration images, realizing an automatic camera calibration process.

[0051] Compared with the prior art, the beneficial effects of the full-automatic camera calibration system and method for adaptive adjustment of brightness and blur provided by the present invention are as follows:

[0052] 1. The full-automatic camera calibration system and method for adaptive adjustment of brightness and blur proposed by the present invention can simplify the camera calibration process. The calibration positions rotate in sequence, eliminating the interference caused by humans and saving a large amount of manpower and costs.

[0053] 2. The full-automatic camera calibration system and method for adaptive adjustment of brightness and blur proposed by the present invention use adaptive blur to judge the image clarity and adaptively adjust the exposure value, with advantages such as strong anti-interference ability, simple operation, and high calculation accuracy.

[0054] 3. The full-automatic camera calibration system and method for adaptive adjustment of brightness and blur proposed by the present invention use the optimal calibration angle and the rotating platform, eliminating the errors and randomness brought by manually adjusting the calibration angle, realizing higher calibration accuracy and stronger robustness. Brief Description of the Drawings

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of a fully automatic camera calibration method for adaptively adjusting brightness and blur in the present invention.

[0057] Figure 2 It is a flowchart of a fully automatic camera calibration system for adaptively adjusting brightness and blur in the present invention. Detailed Description of the Invention

[0058] The following further describes the present invention in detail with reference to embodiments:

[0059] As Figure 1 、 2 shown, for a fully automatic camera calibration system that adaptively adjusts brightness and blur, first, a monocular camera is fixed at a certain position. Below the corresponding monocular camera are a rotary moving platform and a platform motion controller. The rotary moving platform supports a circular calibration board. The initial position of the rotary moving platform is calculated based on the working distance of the camera. The monocular camera captures the current image of the circular calibration board and transmits it back to the automatic high-precision calibration software, and the captured current image is displayed on the display device. The parameters such as the camera gain and acquisition mode are adjusted online through the automatic high-precision calibration software to improve the image quality. The calibration board area of the adjusted image is extracted to obtain a rectangular area, and the size ratio of the black-and-white area is converted to 1:2. The exposure value is adaptively adjusted through the average gray level of the rectangular area. The image with appropriate brightness is subjected to circle detection through the EDcircle algorithm, and the edge gray level points in four directions of the feature circle are extracted. The gray level points are fitted to a curve, and the average slope of the gray level points is calculated to characterize the current blur situation of the image, and it is fed back to the platform controller to control the rotary moving platform to adjust the distance in the Z direction to achieve adaptive adjustment of blur. Furthermore, the platform motion controller controls the rotary moving platform to move to a preset optimal pose group through a preset command. The achievable poses include pitch pose, yaw pose, roll pose, and the distance from the camera (distance in the Z direction). The rotary moving platform supports the calibration board to complete the high-precision calibration of the camera;

[0060] The calibration system includes a monocular camera, a rotary moving platform, a platform motion controller, a circular calibration board, a wireless transmission device, a processor, a display device, and an automatic high-precision calibration software;

[0061] A monocular camera, mainly used to collect the images of the calibration board switched by the rotating mobile platform. The initial position of the camera is ensured to be directly above the rotating mobile platform. In this embodiment, the camera is a Daheng MER-503-20GM-P camera, with a resolution of 2448(H)*2048(V), a frame rate of 20fps, and a data interface of Gige;

[0062] The rotating mobile platform, mainly used to adjust the pose of the center calibration board. In this embodiment, the motor used is an R140 square motor, and the no-load current is 600mA;

[0063] The platform motion controller, mainly used to control the rotating mobile platform to switch poses. By presetting the optimal pose position of the calibration board, the operator sends a start calibration command through the high-precision camera calibration software. The platform motion controller controls the rotating mobile platform to rotate to the preset position one by one. The camera automatically identifies the pose and saves the image according to the switching situation of the rotating mobile platform for calibration calculation. In this embodiment, the stm32 selected is STM32F103ZET6, and the receiving end of the wireless transmission module is a 433 wireless receiving module and a TTL to RS485 module.

[0064] The center calibration board, an image used for camera calibration calculation. By different pose of the center calibration board, the three-dimensional coordinates of the world coordinate system of the calibration board and the two-dimensional coordinates of the image coordinate system are extracted to realize camera calibration. In this embodiment, the center calibration board is made of ceramic material;

[0065] The wireless transmission device, used to send commands to control the platform motion controller, and send corresponding commands to the motion controller according to the operation of the high-precision camera calibration software through the wireless transmission device. In this embodiment, the wireless transmission device used is a 433 wireless transmission module;

[0066] The processor, mainly used to realize data acquisition, storage and internal parameter calibration method, providing a carrier for the internal parameter calibration software; In this embodiment, the processor is a personal PC;

[0067] The display device, mainly used for the operator to operate the high-precision camera calibration software and display the image information collected by the camera in real time; In this embodiment, the display device is a 21-inch LCD display;

[0068] The automatic calibration software includes a function setting area, a function view area, a display view area, and an adaptive blur judgment algorithm; the function setting area is mainly divided into two functions: camera parameter setting and camera calibration. The camera parameter part is mainly used to set the basic parameters of the camera. Under the conditions perceived by the human eye, the camera parameters are set. The main function is to obtain the current camera model in use and display it on the interface; set the camera exposure value, and the exposure value range is 20-1000000; set the gain, and the gain range is not limited; set the automatic white balance function, and automatically identify the white balance function according to the camera type. The white balance option is automatically turned off for black and white cameras. The camera calibration part is mainly used for camera calibration, providing the function of turning the camera on / off to control the running state of the camera; providing a pose adjustment function to control the rotation platform for fine-tuning the pose, setting the rotation speed of the rotation platform, 1-6 preset positions, and the guard position; providing text and image guidance functions, and operating according to the steps shown in the text and images to help the operator quickly understand and master the calibration steps; providing the function of automatically obtaining images, automatically identifying whether the rotation platform has rotated to the preset position according to the state of the rotation platform and obtaining the current stable and clear image; providing an automatic calibration function, performing calibration calculations based on the currently saved calibration images, obtaining data such as camera internal parameters and distortion coefficients, and saving them to a document;

[0069] The function view area is mainly used to display all the functions of the switching part of the function setting area. The function settings are automatically saved to the database, and the content saved last time is automatically read when opened next time, with the function of automatic saving setting;

[0070] The display view area is mainly used to display the images obtained by the current camera, and the effects changed through the function setting area are presented in the display view area.

[0071] A fully automatic camera calibration method for adaptively adjusting brightness and blur includes the following steps:

[0072] Step 1, calculate the initial position of the rotating and moving platform. According to the pinhole imaging model, it is known that the camera has a certain working distance range. Since the focal length of the current camera cannot be known, the focal length f is estimated, and the initial position WD of the rotating and moving platform can be solved through the distance formula;

[0073] Step 2, set the camera parameters. Switch to the camera parameter interface through the function setting area, obtain the camera model and the current image, set the working state of the camera, set the camera acquisition mode and automatic white balance, and manually coarsely adjust the exposure value and camera gain according to the on-site environment to improve the quality of the obtained images; in this embodiment, the obtained camera model is MER-503-20GM-P, the working state of the camera is set to the start acquisition mode, the acquisition mode of the camera is set to the continuous acquisition mode, automatic white balance is prohibited, the coarsely adjusted exposure value is 45000, and the camera gain is set to 1;

[0074] Step 3, adaptively adjust the exposure. The color of the center calibration plate consists of black and white. The calibration plate is placed directly below the lens. Count the proportion R of the number of black and white pixels in a fixed-size rectangular area at the center of the image. n , according to the proportion R of the number of black and white pixels with moderate brightness o for the proportion R in the currently counted fixed area n perform the conversion of the proportion of the number of black and white pixels, and calculate the average gray value G of the current area avec , adaptively adjust the exposure E according to the average gray value;

[0075] The function of step 3 to adaptively adjust the exposure is realized through the following steps:

[0076] Step 3.1, calculate the average gray value in the rectangular area: Select a rectangular area R with a certain area in the center calibration plate area. Set the proportion of the number of standard black and white pixels as R o , calculate the current proportion R of the number of black and white n and based on R o obtain the black and white compensation parameters B c and W c , convert the current proportion of the number of black and white into the same as the set standard value through the compensation parameters, so as to obtain the average gray value after adding the compensation parameters. In this embodiment, the proportion R of the number of standard black and white pixels is defined as o 1:2, and the proportion R of the number of black and white in the currently selected area n is 3:4. Therefore, the compensation parameters B c and W c are 7 / 9 and 7 / 6. Among them, the compensation parameters B c and W c The expressions are:

[0077]

[0078]

[0079] The average gray value G avec The expression formula is:

[0080]

[0081] Step 3.2, adaptively adjust the exposure: Collect a series of brightness images from under-exposure to over-exposure and calculate the average gray value G for each of them avec, construct the brightness distribution based on the average gray value with compensation and weight assignment, calculate the average gray value of the currently selected area, and derive the brightness adjustment strategy based on the brightness distribution to achieve adaptive adjustment of the exposure. The brightness adjustment strategy is as follows:

[0082]

[0083] Step 4, adaptively adjust the blur. By sorting the characteristic circles on the center calibration plate of the circle, obtain the gray values of the four-direction edges of multiple specific characteristic circles and perform curve fitting respectively, calculate the average slope of the gray values in each direction, compare the average slopes in the same direction of multiple circles, and remove the maximum value; calculate the average slope of the edge points in multiple directions of multiple circles as the evaluation factor of the image blur degree, and achieve adaptive adjustment of the blur according to the evaluation factor;

[0084] The function of Step 4 for adaptively adjusting the blur is realized through the following steps:

[0085] Step 4.1, sort the characteristic circles: Detect the characteristic circles in the image, remove the interfering characteristic circles and approximate circles, and sort the characteristic circles, and select multiple specific characteristic circles c1, c2, c3...K as the processing objects. In this embodiment, 3 characteristic circles are selected as the processing objects;

[0086] Step 4.2, edge point evaluation index: Count the edge points of the selected multiple characteristic circles in four directions. Since the gray value gradient of the continuous pixel points of the edge points is large, and the gray value gradient change of the pixel points inside and outside the characteristic circle is small, derive the edge point evaluation index, and filter out the edge pixel points based on this. The expression of the edge point evaluation index is:

[0087]

[0088] Step 4.3, blur evaluation index. According to the pixel values of the edge points in four directions of multiple characteristic circles, perform polynomial fitting curves, calculate the slope of each edge point pixel value, and filter out the average slope with a large difference from the average slope of the edge point pixel values of other characteristic circles in the same direction, so as to calculate the average slope of the edge point pixel values in multiple directions of multiple circles as the evaluation index of the image blur; the expression of the blur evaluation index is:

[0089]

[0090] Step 4.4, adaptively adjust the blur. According to the blur evaluation factor, judge the current image quality, and adjust the distance between the camera and the center calibration plate through the adaptive adjustment strategy of the image blur to improve the image quality; the expression of the adaptive blur adjustment is as follows:

[0091]

[0092] In the case of adaptively adjusting the distance between the camera and the marker, t n represents the evaluation of the change in blur after the current movement, t n is determined according to the following relationship:

[0093] t n = K n - K n-1

[0094] Step 5: Obtain the optimal calibration angle. Rotate the moving platform to preset a set of optimal calibration angles and record them in the platform motion controller. The high-precision camera calibration software sends a start calibration command to the platform motion controller through a wireless transmission device. The rotating moving platform rotates each preset pose one by one and collects stable and clear calibration images at different pose distances through the camera, a total of 18 calibration images;

[0095] The function of Step 5 to obtain the optimal calibration angle is realized through the following steps:

[0096] Step 5.1: Preset the optimal calibration angle. If the inclination angles of pitch, yaw, and roll of the calibration board are too large, the characteristic circle will be distorted. Through repeated experiments and verification, the optimal calibration angle is obtained and the preset optimal calibration angle is recorded in the platform motion controller. In this embodiment, the number of optimal calibration angles is set to 18, which are divided into three groups for collection. Each group has six fixed poses: pitch 5° roll -5°, pitch 5° roll -10°, pitch -5° roll -15°, pitch 5° roll -20°, pitch 10° roll -15°, pitch 10° roll -20°; The three groups of calibration images are divided as follows: the first group, the distance between the camera and the pose rotating table is 26.5 cm; the second group, the distance between the camera and the pose rotating table is 27 cm; the third group, the distance between the camera and the pose rotating table is 27 cm and the pose rotating table has a yaw of 5°;

[0097] Step 5.2: Calibration image acquisition. The automatic high-precision calibration software sends a start calibration command to the platform controller through a wireless transmission device and controls the rotating moving platform to rotate the preset angles in sequence. The automatic high-precision calibration software automatically identifies the state of the rotating moving platform and saves the stable and clear calibration images;

[0098] Step 6: Camera calibration. For each of the 18 calibration images, obtain the center of the characteristic circle and calculate the camera internal parameters and distortion coefficients according to the Zhang Zhengyou calibration method. The data results are generated into a calibration log and saved in the database for the convenience of operators to view and use; In this embodiment, the center distance of the calibration board for the center calibration is 4 mm, and the camera internal parameter matrix calculated through the Zhang Zhengyou calibration method of the camera is:

[0099] [3.5357001241414760e+03,0,1.2380236993640738e+03,0,3.5357001241414760e+03,1.0383782746050902e+03,0,0,1]

[0100] Distortion coefficient of the camera:

[0101] [-5.2923747789825483e-02,-8.5220782269244866e-01,-2.3118926131315680e-06,6.5681051237872253e-04,3.6634495847923979e+01]

[0102] The reprojection error of the camera is 0.015, and the camera calibration accuracy is high;

[0103] The high-precision camera calibration system can adaptively adjust the exposure and blur, improve the quality of the acquired images, move to the preset optimal calibration angles in sequence through the rotating moving platform, and calculate data such as the internal parameters and distortion coefficients of the camera according to the acquired calibration images, realizing an automatic camera calibration process.

[0104] The embodiments described above are only used to describe the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention device.

Claims

1. A fully automatic camera calibration system for adaptively adjusting brightness and blurriness, characterized in that: It consists of a monocular camera, a rotating mobile platform, a platform motion controller, a center calibration board, a wireless transmission device, a processor, a display device, and an automatic high-precision calibration software; this calibration system includes: the monocular camera is fixed at a certain position, and the rotating mobile platform is located below the corresponding camera. The initial value of the distance between the rotating mobile platform and the camera is calculated through the working distance. The platform motion controller drives the calibration board supported by the rotating mobile platform to perform pitching, yawing, and rolling, and adjusts the distance between the calibration board and the camera. The monocular camera takes images of the center calibration board in different poses to complete camera calibration; Below the monocular camera are the rotating mobile platform and the platform motion controller. The rotating mobile platform supports the center calibration board. The initial value of the distance between the rotating mobile platform and the camera is calculated through the working distance. The monocular camera collects the current image of the center calibration board and transmits it back to the automatic high-precision calibration software, and displays the current image on the display device. The automatic high-precision calibration software adjusts the camera gain and acquisition mode parameters online to improve the image quality, extracts the rectangular area of the adjusted image, and converts the size ratio of the black-and-white area to 1:

2. The exposure value is adaptively adjusted through the average gray level of the rectangular area. The image with appropriate brightness is subjected to circle detection by the EDcircle algorithm, and the edge gray points in four directions of the feature circle are extracted. The gray points are fitted to a curve, and the average slope of the gray points is calculated to characterize the blurriness of the image, and it is fed back to the platform controller to control the rotating mobile platform to adjust the Z-direction distance to achieve adaptive adjustment of the blurriness; The platform motion controller controls the rotating mobile platform to adjust to the preset optimal pose group through preset commands. The achieved poses include pitching pose, yawing pose, rolling pose, and adjusting the distance from the camera. The rotating mobile platform supports the calibration board to move to achieve high-precision calibration of the camera.

2. The fully automatic camera calibration system for adaptively adjusting brightness and blur according to claim 1, characterized in that: The monocular camera consists of a camera and a lens, and is used to collect the images of the calibration board switched by the rotating mobile platform. The initial position of the camera is ensured to be directly above the rotating mobile platform; The rotating mobile platform is a four-degree-of-freedom mechanical structure. Each degree of freedom is adjusted by a motor to achieve angle adjustment. The motors are uniformly controlled by the platform motion controller, and can achieve angle switching and adjustment of pitching attitude, yawing attitude, and rolling attitude, and adjust the distance from the camera, and is used to adjust the pose of the center calibration board; The platform motion controller takes the stm32 development board as the core, is connected to the motors on the rotating mobile platform by connecting wires, and the other end of the stm32 is connected to the receiving end of the wireless transmission module. It is used to control the rotating mobile platform to switch poses. Through presetting the optimal pose positions of the calibration board, the operator sends a start calibration command through the high-precision camera calibration software. The platform motion controller controls the rotating mobile platform to rotate to the preset positions one by one. The camera automatically identifies the poses according to the switching situation of the rotating mobile platform and saves the images for calibration calculation; The center calibration board is an image used for camera calibration calculation. Through different poses of the center calibration board, the three-dimensional coordinates of the world coordinate system and the two-dimensional coordinates of the image coordinate system of the calibration board are extracted to achieve camera calibration; A wireless transmission device, which is used to send commands to the motion controller of the control platform and send corresponding commands to the motion controller through the wireless transmission device according to the operation of the high-precision camera calibration software; A processor, which is used to implement data acquisition, storage, and internal parameter calibration methods and provide a carrier for the internal parameter calibration software; A display device, which is used for operators to operate the high-precision camera calibration software and display the image information collected by the camera in real time; The automatic calibration software includes a function setting area, a function view area, a display view area, and a high-precision calibration algorithm; Among them, the function setting area provides two functions required for camera calibration, namely camera parameter setting and camera calibration. Operators can select the functions according to the actual situation; the function view area specifically displays the selected calibration function; the display view area is used to display the images obtained by the camera.

3. A fully automatic camera calibration method for adaptively adjusting brightness and blur, including the following steps: Step 1, calculate the initial position of the rotary moving platform. According to the pinhole imaging model, it is known that the camera has a certain working distance range. Since the focal length of the current camera is unknown, the initial position WD of the rotary moving platform can be solved through the distance formula by estimating the focal length f; Step 2, set the camera parameters. Switch to the camera parameter interface through the function setting area, obtain the model of the camera and the current image, set the working state of the camera, set the camera acquisition mode and automatic white balance, and manually coarsely adjust the exposure value and camera gain according to the on-site environment to improve the quality of the obtained images; Step 3, adaptively adjust the exposure. The color composition of the center calibration plate is black and white. The calibration plate is directly below the lens. Statistically calculate the ratio R of the number of black and white pixels in a fixed-size rectangular area at the center of the image. n , according to the ratio R of the number of black and white pixels with moderate brightness o For the ratio R in the currently statistically fixed area n perform a conversion of the ratio of the number of black and white pixels, and calculate the average gray value of the current area. Adaptively adjust the exposure according to the average gray value; Step 4, adaptively adjust the blur. By sorting the feature circles on the center calibration plate, obtain the four-direction edge gray values of multiple specific feature circles and perform curve fitting respectively, calculate the average slope of the gray values in each direction, compare the average slope values in the same direction of multiple circles and remove the maximum value; calculate the average slope of the edge points in multiple directions of multiple circles as the evaluation factor of the image blur degree, and adaptively adjust the blur according to the evaluation factor; Step 5, obtain the best calibration angle. Preset a set of best calibration angles on the rotary moving platform and record them in the platform motion controller. The high-precision camera calibration software sends a start calibration instruction to the platform motion controller through the wireless transmission device. The rotary moving platform rotates the preset poses one by one, and the camera collects 18 stable and clear calibration images at different pose distances; Step 6, camera calibration. Obtain the centers of the feature circles for the 18 calibration images respectively and calculate the camera internal parameters and distortion coefficients according to the Zhang Zhengyou calibration method. The data results are generated into a calibration log and saved in the database for operators to view and use.

4. The fully automatic camera calibration method for adaptively adjusting brightness and blur according to claim 3, wherein In Step 3, the adaptive adjustment of the exposure is realized through the following steps: Step 3.1, calculate the average gray value within the rectangular area: Select a rectangular area of a certain area within the circular center calibration plate area, and set the quantity ratio of standard black and white to be R o , calculate the current quantity ratio R of black and white n and based on R o obtain the black and white compensation parameters B c and W c , convert the current quantity ratio of black and white to be consistent with the set standard value through the compensation parameters, so as to obtain the average gray value after adding the compensation parameters; where the compensation parameters B c and W c have the following expressions: Among them, b and b n represent the number of pixels with a value of the standard ratio and black in the nth specific area, and w and w n represent the number of pixels with a value of the standard ratio and white in the nth specific area; Average gray value G avec The expression formula is: Among them, A b represents the weight coefficient of the black area, and A w represents the weight coefficient of the white area. Since the gray value of the black area is much smaller than that of the white area, increasing the weight coefficient A b of the black area results in a more significant change in the average gray value when the brightness changes; Step 3.2, adaptively adjust the exposure: Collect a series of brightness images from underexposure to overexposure and calculate the average gray value G for each image respectively avec , construct a brightness distribution based on the average gray value with compensation and weight assignment, calculate the average gray value of the currently selected area, and derive a brightness adjustment strategy based on the brightness distribution to achieve adaptive exposure adjustment; the brightness adjustment strategy is as follows: Among them, E represents the modified exposure value, L represents the brightness situation, and K L represents the exposure adjustment coefficient, and the currently obtained average gray value G of the region avec , G up and G under represent the average gray values of the upper and lower limits of suitable brightness within the rectangular region; by correcting the E value, the brightness can be adaptively adjusted to obtain a clear calibration image.

5. The fully automatic camera calibration method for adaptively adjusting brightness and blur according to claim 3, characterized in that, In Step 4, the adaptive adjustment of the blur is realized through the following steps: Step 4.1, feature circle sorting: Detect the feature circles in the image, remove the interfering feature circles and approximate circles, sort the feature circles, and select multiple specific feature circles as the processing objects; Step 4.2, Edge Point Evaluation Index: Count the edge points of multiple selected feature circles in four directions. Based on the different gray value change characteristics of the edge points, the internal pixel points, and the external pixel points of the feature circles, derive the edge point evaluation index, and filter out the edge pixel points based on this; the expression of the edge point evaluation index is: Among them, M n represents the evaluation index of whether the nth point is an edge point, and G n represents the gray value of the nth point, and d represents the step size; Step 4.3, Blur Evaluation Index: Perform polynomial fitting on the edge point pixel values in four directions of multiple feature circles, calculate the slope of each edge point pixel value, and filter out the average slope with a large difference from the average slope of the edge point pixel values of other feature circles in the same direction, so as to calculate the average slope of the edge point pixel values in multiple directions of multiple circles as the evaluation index of image blur; the expression of the blur evaluation index is: Among them, m is the number of edge points, Y is the expression of the edge point pixel values after polynomial fitting, z represents the direction of the edge points, and i represents the number of feature circles obtained. Step 4.4, Adaptive Adjustment of Blur: Judge the current image quality according to the blur evaluation factor, and adjust the distance between the camera and the center calibration board through the adaptive image blur adjustment strategy to improve the image quality; the expression of the adaptive blur adjustment is as follows: Among them, D n represents the camera moving distance direction, A k and K o represent the fixed coefficient and the image sharpness threshold, t n represents the direction weight. In the adaptive blur adjustment of the distance between the camera and the marker, t n represents the evaluation of the change in blur after the current movement, and t n is determined according to the following relationship: t n = K n -K n-1 Among them, K n represents the blur evaluation factor of the nth image.

6. The fully automatic camera calibration method for adaptively adjusting brightness and blur according to claim 3, characterized in that In Step 5, the following steps are included: Step 5.1, preset the optimal calibration angle: If the inclination angles of the calibration board in pitch, yaw, and roll are too large, the feature circles will be distorted. Through repeated experiments, the optimal calibration angle is obtained and recorded in the platform motion controller. The number of optimal calibration angles is set to 18, which are collected in three groups, and each group has six fixed poses: pitch 5°, roll -5°; pitch 5°, roll -10°; pitch -5°, roll -15°; pitch 5°, roll -20°; pitch 10°, roll -15°; pitch 10°, roll -20°. The three groups of calibration images are divided as follows: for the first group, the distance between the camera and the pose rotating table is WD + D n , for the second group, the distance between the camera and the pose rotating table is WD + D n + WD1, and for the third group, the distance between the camera and the pose rotating table is WD + D n + WD1 and the pose rotating table is yawed by 5°, where D n represents the distance moved for adaptive adjustment of the blur, and WD1 represents one-fifth of the current lens depth of field; Step 5.2, Calibration Image Acquisition: The automatic high-precision calibration software sends a start calibration command to the platform controller through the wireless transmission device and controls the rotation and movement platform to rotate by a preset angle in sequence. The automatic high-precision calibration software automatically identifies the state of the rotation and movement platform and saves the stable and clear calibration images; the high-precision camera calibration system can adaptively adjust the exposure and the blur, improve the quality of the acquired images, move the rotation and movement platform to the preset optimal calibration angles in sequence, and calculate the camera internal parameters and distortion coefficient data based on the acquired calibration images to achieve the automatic camera calibration process.

Images