A method and device for adaptively adjusting camera exposure parameters and marker recognition thresholds

Abstract

The present invention discloses a method and device for adaptively adjusting camera exposure parameters and marker recognition thresholds, the method comprising the steps of: setting the initial exposure parameters and initial recognition thresholds of a binocular camera, acquiring left and right eye images, and extracting the center of the marker in the left and right eye images respectively; reconstructing the three-dimensional point of the marker in the world coordinate system according to the internal and external parameters of the binocular camera; calculating and sorting the distance between the binocular camera and the three-dimensional point, and taking the minimum distance value as the reference distance; substituting the reference distance into the pre-fitted distance exposure equation and exposure threshold equation to calculate the exposure parameters and recognition threshold; adaptively adjusting the binocular camera according to the calculated exposure parameters and recognition threshold. The present invention can automatically adjust the appropriate camera exposure parameters, and automatically determine the recognition threshold under the current distance and exposure parameters, effectively expanding the effective working space, and improving the positioning accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a method and device for adaptively adjusting camera exposure parameters and marker recognition thresholds. Background Art

[0002] At present, one of the implementation methods of high-precision binocular near-infrared positioning technology is to first identify the circular marker, and then align the circular marker according to the preset rules to determine the target position and posture. Before the circular marker is recognized, the marker needs to be imaged with high quality, and the imaging and recognition of the marker strongly depend on the exposure parameters of the camera and the recognition threshold corresponding to the exposure parameters. Existing methods often give a fixed exposure parameter and recognition threshold for imaging and positioning calculations of the marker. In actual operation, when the position of the marker from the camera changes, the brightness of the marker's image will fluctuate with the distance, and the rule is: the farther the marker is from the camera, the darker the image, and vice versa. Therefore, the existing method is not suitable for use in a large range of working scenes, resulting in limitations in the positioning area and even directly affecting the positioning accuracy. Summary of the invention

[0003] Technical purpose: In response to the above technical problems, the present invention proposes a method and device for adaptively adjusting camera exposure parameters and marker recognition thresholds, which can automatically adjust appropriate camera exposure parameters and automatically determine the recognition threshold under the current distance and exposure parameters, effectively expanding the effective working space and improving positioning accuracy.

[0004] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0005] A method for adaptively adjusting camera exposure parameters and marker recognition thresholds for a binocular camera, characterized in that it comprises the steps of:

[0006] Set the initial exposure parameters and initial recognition threshold of the binocular camera, obtain the left eye image and the right eye image at the same time, and extract the centers of all markers in the left eye image and the right eye image respectively using the initial recognition threshold;

[0007] Reconstruct the 3D points of all markers in the world coordinate system based on the intrinsic and extrinsic parameters of the binocular camera;

[0008] Calculate the distance between the binocular camera and the 3D points and sort them, taking the smallest distance value as the reference distance;

[0009] Substituting the reference distance into the pre-fitted distance exposure equation f(d) and exposure threshold equation g(d), respectively, to calculate the exposure parameter exp and the recognition threshold threshold;

[0010] The stereo camera is adaptively adjusted according to the calculated exposure parameter exp and threshold threshold.

[0011] Preferably, the distance exposure equation f(d) and the exposure threshold equation g(d) are obtained by the following steps:

[0012] (1) Determine the working range of the binocular camera and set multiple sampling working distances within the working range;

[0013] (2) Determine the background noise threshold in the imaging, that is, set the exposure parameter of the binocular camera to 0, adjust the infrared fill light power to the maximum value, capture an image without a reflective background, and count the pixel values ​​in the image, draw a three-dimensional statistical graph, and use the highest value in the three-dimensional statistical graph as the background noise threshold;

[0014] (3) The binocular camera automatically exposes at each selected sampling working distance, and records the experimental exposure parameters and experimental recognition thresholds when the images taken at each sampling working distance meet the preset exposure conditions;

[0015] (4) lowering the experimental exposure parameter N times according to a preset step length, and segmenting the markers using M thresholds of the grayscale of the adjacent edges, and obtaining the coordinates of the center of the marker circle using an edge fitting algorithm, that is, obtaining the coordinates of M*(N+1) center of circles at one sampling working distance;

[0016] (5) Evaluate the obtained M*(N+1) circle centers, select the best circle center coordinates, and record the experimental exposure parameters and experimental recognition thresholds during automatic exposure corresponding to the best circle center coordinates;

[0017] (6) Repeating steps (4) to (5) at the set sampling working distance to obtain multiple sets of corresponding optimal circle center coordinates, experimental exposure parameters, and experimental recognition thresholds;

[0018] (7) According to the optimal circle center coordinates, experimental exposure parameters and experimental recognition threshold, the distance exposure equation f(d) and the exposure threshold equation g(d) are fitted, and the distance exposure curve and the exposure threshold curve are plotted.

[0019] Preferably, the M*(N+1) circle centers are evaluated in the following steps:

[0020] The area where the circular marker is located is used as a target;

[0021] Calculate the coordinates of the circle center multiple times to obtain multiple corresponding circle centers;

[0022] The centroid of all circle centers is calculated as the ideal bull's-eye, and the coordinates of the circle center closest to the bull's-eye are defined as the optimal circle center coordinates.

[0023] A device for adaptively adjusting camera exposure parameters and marker recognition thresholds, comprising:

[0024] The image acquisition and marker extraction module is used to set the initial exposure parameters and initial recognition threshold of the binocular camera, acquire the left eye image and the right eye image at the same time, and extract the centers of all markers in the left eye image and the right eye image respectively with the initial recognition threshold;

[0025] The 3D point coordinate determination module is used to reconstruct the 3D points of all markers in the world coordinate system based on the internal and external parameters of the binocular camera;

[0026] The reference distance calculation module is used to calculate the distance between the binocular camera and the three-dimensional points and sort them, and take the smallest distance value as the reference distance;

[0027] A calculation module, used for respectively substituting the reference distance into the pre-fitted distance exposure equation f(d) and exposure threshold equation g(d), and calculating the exposure parameter exp and the recognition threshold threshold;

[0028] The adaptive adjustment module is used to adaptively adjust the binocular camera according to the calculated exposure parameter exp and threshold threshold;

[0029] The fitting module is used to fit the distance exposure equation f(d) and the exposure threshold equation g(d), and the curves corresponding to the equations.

[0030] An electronic device, characterized in that the electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded by the processor and executes the method.

[0031] A computer-readable storage medium, characterized in that it is used to store a computer program; wherein the computer program is used by a processor to execute the method.

[0032] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0033] The present invention can automatically adjust appropriate camera exposure parameters according to the position of the marker in the effective working space, and automatically determine the recognition threshold under the current distance and exposure parameters, which can effectively expand the effective working space and improve the positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a method for adaptively adjusting camera exposure parameters and marker recognition thresholds proposed in an embodiment;

[0035] Figure 2 It is a three-dimensional statistical graph;

[0036] Figure 3 Schematic diagram of camera parameter settings;

[0037] Figure 4 For the corresponding Figure 3 The “distance-exposure” curve and “exposure-threshold” curve of the parameters;

[0038] Figure 5 Flowchart of a method for adaptively adjusting camera exposure parameters and marker recognition thresholds. DETAILED DESCRIPTION

[0039] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0040] Embodiment 1

[0041] This embodiment proposes a method for adaptively adjusting camera exposure parameters and marker recognition thresholds, such as Figure 1 As shown, the steps include:

[0042] Set the initial exposure parameters and initial recognition threshold of the binocular camera, obtain the left eye image and the right eye image at the same time, and extract the centers of all markers in the left eye image and the right eye image respectively using the initial recognition threshold;

[0043] Reconstruct the 3D points of all markers in the world coordinate system based on the intrinsic and extrinsic parameters of the binocular camera;

[0044] Calculate the distance between the binocular camera and the 3D points and sort them, taking the smallest distance value as the reference distance;

[0045] Substituting the reference distance into the pre-fitted distance exposure equation f(d) and exposure threshold equation g(d), respectively, to calculate the exposure parameter exp and the recognition threshold threshold;

[0046] The stereo camera is adaptively adjusted according to the calculated exposure parameter exp and threshold threshold.

[0047] Specifically, first, set a default initial exposure parameter, obtain two images of the left and right eyes, and extract the center of the marker in the left and right eye images respectively with the default threshold. Then, according to the binocular epipolar geometric constraint relationship, find the corresponding point relationship between the left and right eyes, and reconstruct the three-dimensional spatial position of the marker based on the intrinsic and extrinsic parameters of the binocular camera. Secondly, in order to ensure that all points in the scene are not over-bright, the distance between the closest point to the camera and the camera is often selected as a reference for subsequent exposure and threshold. Referring to the direction of the coordinate system established previously, that is, in the right-handed coordinate system established with the midpoint of the baseline of the binocular camera as the origin, the baseline as the y-axis, and the plane perpendicular to the front shell of the camera as the z-axis, find the three-dimensional point closest to the camera according to the coordinates of the three-dimensional point, and record the distance d between the point and the camera.

[0048] The distance d is substituted into the pre-fitted distance-exposure equation f(d) and exposure-threshold equation g(d) to obtain the exposure parameter exp and the threshold threshold, and the camera is adaptively adjusted according to the exposure parameter exp and the threshold threshold.

[0049] Among them, the distance-exposure equation f(d) and the exposure-threshold equation g(d) are obtained according to the following steps:

[0050] S1. Determine the working distance range of the binocular camera. Generally, the exposure of the camera is related to the distance. When the marker to be measured is far away from the camera, the image will become darker. If effective imaging is required, the exposure should be appropriately increased.

[0051] S2. Determine the background noise threshold in the image. Since most of the pixels in the image are background, they are meaningless for recognition. Set the exposure parameter to 0, adjust the infrared fill light power to the maximum, shoot without reflective background, and count the pixel values ​​to make a three-dimensional statistical graph, see Figure 2 The background noise threshold can be set to the highest value, or slightly higher than the highest value. In this embodiment, it is set to slightly higher than the highest value to leave redundancy for abnormal points in the imaging. The three-dimensional statistical graph is used to determine the impact of the camera background noise. Its purpose is to determine the background noise threshold. In order to filter out the background noise, a three-dimensional statistical graph is made, and the highest value in the three-dimensional statistical graph is used as the threshold for filtering out the background noise.

[0052] S3. Set a suitable distance space for sampling to obtain the relationship between distance and exposure parameters. It should be noted that in order to present better edge details of the marker in the image, the highest grayscale value of the marker is defined as no more than 250 as appropriate exposure. The sampling steps are as follows:

[0053] S3.1. Determine a distance, and adjust the exposure to the appropriate level under the above conditions according to the brightness of the reflective ball in the image, and record the exposure parameters at this time.

[0054] S3.2, the exposure parameters obtained in S3.1 are lowered N times at a fixed step length, and the marker is segmented by M thresholds of the grayscale of the adjacent edge, and the coordinates of the center of the marker circle are obtained by the edge fitting algorithm, that is, at one sampling working distance, M*(N+1) coordinates of the center of the circle are obtained; if the step length is 20, different values ​​can be taken for different cameras. And according to the grayscale value of the edge pixel of the reflective ball, 16 thresholds of the grayscale of the adjacent edge are used to segment the marker, and the coordinates of the center of the marker circle are obtained by the edge fitting algorithm, that is, at one distance, 16*4=64 coordinates of the center of the circle are obtained.

[0055] S3.3. Evaluate the positioning accuracy of the 64 circle centers obtained in S3.2, select the best circle center coordinates, and record the exposure parameters and recognition threshold at this time.

[0056] S3.4 fits the “distance-exposure” equation and curve, and the “exposure-recognition threshold” equation and curve according to the distance, exposure, and recognition threshold recorded in S3.2 and S3.3.

[0057] The evaluation method of the center coordinates in S3.3 is as follows:

[0058] The area where the circular marker is located is used as a "target";

[0059] Calculating the coordinates of the center of a circle is the process of "target shooting". The more concentrated the points are, the more accurate the target shooting is, and vice versa.

[0060] The center of mass of all circle centers can be calculated as the ideal "bull's eye", and the center of the circle closest to the "bull's eye" is defined as the optimal center coordinate.

[0061] The method of this embodiment is described in detail by taking the exposure parameter of the camera as an example.

[0062] 1. Experimental process:

[0063] 1. Working range: 900mm-2400mm

[0064] 2. Set the sampling working distance: 900mm, 1100mm, 1300mm, 1500mm, 1700mm, 1900mm, 2100mm, 2300mm, 2400mm.

[0065] Step 1), let the camera automatically expose at the preset working distance, make the image just overexposed, and record the exposure parameters at this time. Automatic exposure is achieved using software, and the specific implementation method is as follows:

[0066] i. Set a threshold slightly lower than the edge of the marker. Areas above this threshold are considered valid marker areas. And judge whether the brightness of each marker is appropriate. The marker is the circular marker mentioned above. There can be multiple markers. In this example, 4 markers are used.

[0067] ii. If the marker brightness is too high or too low, adjust the exposure parameter down or up in steps of 5.

[0068] iii. Until the marker brightness is appropriate, record the exposure parameters at this time.

[0069] Step 2), lower the exposure parameter obtained in 1) by 20 steps for 3 times. Each time the exposure parameter is lowered, the exposure and threshold parameters are reduced, and 16 thresholds of 40-55 are used to identify the (circular) marker. That is, at one distance, after changing the exposure parameter once, 16 center coordinates are obtained for 16 thresholds; after changing it 3 times and adding the initial value once, 16*4=64 center coordinates are obtained.

[0070] Step 3), evaluate the quality of the 64 circle centers obtained in step 2), select the best circle center coordinates, and record the exposure parameters and recognition threshold at this time.

[0071] Step 4), repeat steps 1) to 3) at the set sampling working distance, and fit the distance-exposure equation f(d) and exposure-threshold equation g(d) according to the obtained multiple sets of exposure parameters and recognition thresholds, and make the "distance-exposure" curve and "exposure-threshold" curve, see Figure 4 .

[0072] 2. Usage process:

[0073] After obtaining the "distance-exposure" curve and the "exposure-threshold" curve, you can use the two curves to perform adaptive parameter adjustment when the camera adaptively adjusts the exposure parameters and the marker recognition threshold. Figure 5 . The steps include:

[0074] 1. Read the left and right camera images;

[0075] 2. Extract the sub-pixel coordinates of the marker points in the left and right images. The marker points are the markers mentioned above.

[0076] 3. Calculate the three-dimensional coordinates of the marker points based on the internal and external parameters of the binocular camera;

[0077] 4. Calculate the distance from the camera center to the marked point and sort them to get the minimum distance d;

[0078] 5. Substitute the distance d into the distance-exposure equation f(d) and exposure-threshold equation g(d) fitted in step 4) to obtain the exposure parameter exp and the threshold threshold;

[0079] 6. Set the exposure parameter exp and the threshold parameter threshold;

[0080] 7. End.

[0081] Embodiment 2

[0082] This embodiment provides a device for adaptively adjusting camera exposure parameters and marker recognition thresholds, which is used to implement the method described in the first embodiment, including:

[0083] The image acquisition and marker extraction module is used to set the initial exposure parameters and initial recognition threshold of the binocular camera, acquire the left eye image and the right eye image at the same time, and extract the centers of all markers in the left eye image and the right eye image respectively with the initial recognition threshold;

[0084] The 3D point coordinate determination module is used to reconstruct the 3D points of all markers in the world coordinate system based on the internal and external parameters of the binocular camera;

[0085] The reference distance calculation module is used to calculate the distance between the binocular camera and the three-dimensional points and sort them, and take the smallest distance value as the reference distance;

[0086] A calculation module, used for respectively substituting the reference distance into the pre-fitted distance exposure equation f(d) and exposure threshold equation g(d), and calculating the exposure parameter exp and the recognition threshold threshold;

[0087] The adaptive adjustment module is used to adaptively adjust the binocular camera according to the calculated exposure parameter exp and threshold threshold;

[0088] And, a fitting module is used to fit the distance exposure equation f(d) and the exposure threshold equation g(d), and the curves corresponding to the equations.

[0089] In yet another embodiment of the present invention, an electronic device is provided, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the method.

[0090] In yet another embodiment of the present invention, a computer-readable storage medium is provided for storing a computer program; wherein the method is implemented when the computer program is executed by a processor.

[0091] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for adaptively adjusting camera exposure parameters and marker recognition thresholds for a binocular camera, characterized in that: Includes steps: Set the initial exposure parameters and initial recognition threshold of the binocular camera, obtain the left eye image and the right eye image at the same time, and extract the centers of all markers in the left eye image and the right eye image respectively using the initial recognition threshold; Reconstruct the 3D points of all markers in the world coordinate system based on the intrinsic and extrinsic parameters of the binocular camera; Calculate the distance between the binocular camera and the 3D points and sort them, taking the smallest distance value as the reference distance; Substituting the reference distance into the pre-fitted distance exposure equation f(d) and exposure threshold equation g(d), respectively, to calculate the exposure parameter exp and the recognition threshold threshold; Adaptively adjust the binocular camera according to the calculated exposure parameter exp and recognition threshold threshold; The distance exposure equation f(d) and the exposure threshold equation g(d) are obtained by the following steps: (1) Determine the working range of the binocular camera and set multiple sampling working distances within the working range; (2) Determine the background noise threshold in the imaging. Specifically, set the exposure parameter of the binocular camera to 0, adjust the infrared fill light power to the maximum value, capture an image without a reflective background, and count the pixel values ​​in the image to draw a three-dimensional statistical graph. The highest value in the three-dimensional statistical graph is used as the background noise threshold. (3) The binocular camera automatically exposes at each selected sampling working distance and records the experimental exposure parameters and experimental recognition thresholds when the images taken at each sampling working distance meet the preset exposure conditions; (4) lowering the experimental exposure parameter N times according to a preset step length, and segmenting the marker using M thresholds of the grayscale of the adjacent edge, and obtaining the coordinates of the center of the marker circle using an edge fitting algorithm, specifically obtaining the coordinates of M*(N+1) centers of the circle at one sampling working distance; (5) Evaluate the M*(N+1) circle centers, select the best circle center coordinates, and record the experimental exposure parameters and experimental recognition thresholds during automatic exposure corresponding to the best circle center coordinates; (6) Repeat steps (4) to (5) at the set sampling working distance to obtain multiple sets of corresponding optimal circle center coordinates, experimental exposure parameters, and experimental recognition thresholds; (7) According to the optimal circle center coordinates, experimental exposure parameters and experimental recognition threshold, the distance exposure equation f(d) and the exposure threshold equation g(d) are fitted, and the distance exposure curve and the exposure threshold curve are plotted.

2. The method for adaptively adjusting camera exposure parameters and marker recognition thresholds according to claim 1, characterized in that: Evaluate the obtained M*(N+1) circle centers in the following steps: The centroid of all circle centers is calculated as the ideal bull's-eye, and the coordinates of the circle center closest to the bull's-eye are defined as the optimal circle center coordinates.

3. A device for adaptively adjusting camera exposure parameters and marker recognition thresholds, characterized in that: include: The image acquisition and marker extraction module is used to set the initial exposure parameters and initial recognition threshold of the binocular camera, acquire the left eye image and the right eye image at the same time, and extract the centers of all markers in the left eye image and the right eye image respectively with the initial recognition threshold; The 3D point coordinate determination module is used to reconstruct the 3D points of all markers in the world coordinate system based on the internal and external parameters of the binocular camera; The reference distance calculation module is used to calculate the distance between the binocular camera and the three-dimensional points and sort them, and take the smallest distance value as the reference distance; A calculation module, used for substituting the reference distance into the pre-fitted distance exposure equation f(d) and exposure threshold equation g(d), respectively, to calculate the exposure parameter exp and the recognition threshold threshold; The adaptive adjustment module is used to adaptively adjust the binocular camera according to the calculated exposure parameter exp and recognition threshold threshold; A fitting module, used for fitting the distance exposure equation f(d) and the exposure threshold equation g(d), and the curves corresponding to the equations; The distance exposure equation f(d) and the exposure threshold equation g(d) are obtained by the following steps: (1) Determine the working range of the binocular camera and set multiple sampling working distances within the working range; (2) Determine the background noise threshold in the imaging. Specifically, set the exposure parameter of the binocular camera to 0, adjust the infrared fill light power to the maximum value, capture an image without a reflective background, and count the pixel values ​​in the image to draw a three-dimensional statistical graph. The highest value in the three-dimensional statistical graph is used as the background noise threshold. (3) The binocular camera automatically exposes at each selected sampling working distance and records the experimental exposure parameters and experimental recognition thresholds when the images taken at each sampling working distance meet the preset exposure conditions; (4) lowering the experimental exposure parameter N times according to a preset step length, and segmenting the marker using M thresholds of the grayscale of the adjacent edge, and obtaining the coordinates of the center of the marker circle using an edge fitting algorithm, specifically obtaining the coordinates of M*(N+1) centers of the circle at one sampling working distance; (5) Evaluate the M*(N+1) circle centers, select the best circle center coordinates, and record the experimental exposure parameters and experimental recognition thresholds during automatic exposure corresponding to the best circle center coordinates; (6) Repeat steps (4) to (5) at the set sampling working distance to obtain multiple sets of corresponding optimal circle center coordinates, experimental exposure parameters, and experimental recognition thresholds; (7) According to the optimal circle center coordinates, experimental exposure parameters and experimental recognition threshold, the distance exposure equation f(d) and the exposure threshold equation g(d) are fitted, and the distance exposure curve and the exposure threshold curve are plotted.

4. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein the computer program implements the method according to any one of claims 1 to 2 when executed by a processor.

Images