Visual automatic calibration system and method and storage medium

Abstract

The invention relates to the technical field of vision measurement, in particular to an automatic vision calibration system and method and a storage medium, the automatic vision calibration system comprises a vision measuring instrument, an underwater self-generating calibration plate, a control unit and a power supply unit, the underwater self-generating calibration plate and the power supply unit are connected with the control unit, and the control unit is connected with the underwater self-generating calibration plate. The power supply unit is connected with the underwater self-generating cursor fixing plate, and the vision measuring instrument faces the underwater self-generating cursor fixing plate; the underwater self-luminous calibration plate comprises a calibration plate body, a plurality of underwater LED units are arranged on the calibration plate body in a matrix mode, the light emitting wavelength of each underwater LED unit is 530 nm, the surface of each underwater LED unit is designed to be flat, and light intensity distribution is consistent. Compared with the prior art, the visual automatic calibration system and method and the storage medium are efficient and accurate, the camera calibration process is simplified into network learning through data-driven calibration, and compared with a traditional underwater calibration method, the calibration precision is high, and the complexity is reduced by eliminating the understanding requirement for an imaging model; the designed underwater self-luminous calibration target system effectively reduces the influence of water body scattering, and is combined with a proposed underwater light spot extraction method to improve the calibration precision.

Description

[Technical field]

[0001] The present invention relates to the field of visual measurement technology, and in particular to a visual automatic calibration system, method and storage medium. [Background technology]

[0002] Underwater stereo vision technology provides an efficient solution for scene perception in aquatic environments. With the increasing popularity of consumer-grade cameras, underwater applications using stereo vision are becoming more and more popular, benefiting various disciplines. In fact, many fields, such as underwater structure inspection, 3D reconstruction in archaeology, and state estimation of autonomous underwater vehicles, require higher accuracy. Therefore, the demand for advanced underwater stereo camera perception technology is increasing.

[0003] Camera calibration is the key to stereo vision, and the accuracy of the calibration results is crucial to subsequent expansion and application. The underwater environment is complex and diverse, and various unfavorable factors will seriously affect the quality of underwater imaging. Scattering and absorption in water will cause image quality to deteriorate, resulting in blurred images and reduced field of view, which will affect the calibration accuracy. [Summary of the invention]

[0004] In order to overcome the above problems, the present invention proposes a visual automatic calibration system, method and storage medium that can effectively solve the above problems.

[0005] A technical solution provided by the present invention to solve the above technical problems is: to provide a visual automatic calibration system including a visual measuring instrument, an underwater self-luminous calibration board, a control unit and a power supply unit, wherein the underwater self-luminous calibration board and the power supply unit are respectively connected to the control unit, the power supply unit is connected to the underwater self-luminous calibration board, and the visual measuring instrument faces the underwater self-luminous calibration board; the underwater self-luminous calibration board includes a calibration board body, on which a plurality of underwater LED units are arranged in a matrix, the light emission wavelength of the underwater LED unit is 530nm, the surface is designed to be flat, and the light intensity distribution is consistent.

[0006] Preferably, there are 130 underwater LED units in total, which are arranged in a matrix of 10 rows and 13 columns, with a spacing of 30 mm between adjacent underwater LED units. A spherical LED lamp is added in the 3rd row and 2nd column to locate the position of the calibration plate.

[0007] Preferably, the visual measuring instrument includes two underwater stereo cameras, and the underwater stereo cameras are Daheng MER2-2000-6GM industrial cameras.

[0008] Preferably, the visual automatic calibration system further comprises an X-moving module and a Y-moving module, wherein the Y-moving module is vertically connected to the X-moving module, the underwater stereo camera is mounted on the Y-moving module through a bracket, and the underwater stereo camera can move along the X and Y directions; the X-moving module and the Y-moving module are respectively connected to the controller.

[0009] Preferably, the automatic visual calibration system further comprises a rotating platform, the underwater self-luminous calibration plate is connected to the rotating platform, and the rotating platform can control the rotation of the underwater self-luminous calibration plate; the rotating platform is connected to a controller.

[0010] A visual automatic calibration method is applied to the visual automatic calibration system, and the visual automatic calibration method comprises the following steps:

[0011] Step S1, placing the underwater self-luminous calibration plate and the vision measuring instrument in water, taking the calibration target that is closest to the vision measuring instrument and can be clearly imaged as the initial calibration plane, and taking the first corner point in the lower left corner of the initial calibration plane as the origin of the world coordinate system;

[0012] Step S2, two underwater stereo cameras collect multiple sets of underwater self-luminous calibration plate image information from different angles;

[0013] Step S3, using a neural network to express the mapping relationship between pixel coordinates and the 3D world plane;

[0014] Step S4, extracting image features by using an underwater spot center extraction method;

[0015] Step S5, generating a calibration result based on all the image data after feature extraction.

[0016] Preferably, in step S3, the image data collected by the two underwater stereo cameras are fused; for X and Y coordinate regression, the mapping between the monocular camera pixel coordinates and the world coordinate plane is retained, and the predicted values ​​of the X and Y coordinates are obtained from the two underwater stereo cameras.

[0017] Preferably, in step S4, the underwater light spot is divided into a core area, an edge area and a background area according to the number of pixels of each brightness level in the light spot image; the average grayscale values ​​of the core area, the edge area and the background area are calculated respectively, and the centroid is extracted by the grayscale centroid method, and the image coordinates and their respective grayscale values ​​are used as weights to determine the center of the light spot by calculating the weighted average.

[0018] Preferably, in step S3, the neural network is a 3D coordinate regression network, and a fully connected neural network (FCNN) is used to regress the Z coordinate; the fully connected neural network (FCNN) architecture includes an input layer, a hidden layer and an output layer, and each layer is connected by weights.

[0019] A computer-readable storage medium stores a computer program, which implements the steps of the visual automatic calibration method when executed by a processor.

[0020] Compared with the prior art, the visual automatic calibration system and method of the present invention are efficient and accurate. The data-driven calibration simplifies the camera calibration process into network learning. Compared with the traditional underwater calibration method, the calibration accuracy is high and the complexity is reduced by eliminating the requirement for understanding the imaging model. The designed underwater self-luminous calibration target system effectively reduces the influence of water scattering and is combined with the proposed underwater light spot extraction method to improve the calibration accuracy.

Brief Description of the Drawings

[0021] Figure 1 It is a framework diagram of the automatic visual calibration system of the present invention;

[0022] Figure 2 Schematic diagram of an underwater self-luminous calibration plate of the automatic visual calibration system of the present invention; [Specific implementation method]

[0023] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0024] It should be noted that in the embodiments of the present invention, all directional indications (such as up, down, left, right, front, back, etc.) are limited to relative positions on the specified view, rather than absolute positions.

[0025] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0026] See also Figure 1 and Figure 2 The automatic visual calibration system of the present invention includes a visual measuring instrument, an underwater self-luminous calibration plate, a control unit and a power supply unit. The underwater self-luminous calibration plate and the power supply unit are respectively connected to the control unit, the power supply unit is connected to the underwater self-luminous calibration plate, and the visual measuring instrument faces the underwater self-luminous calibration plate.

[0027] The visual measuring instrument includes two underwater stereo cameras, and the underwater stereo cameras adopt Daheng MER2-2000-6GM industrial cameras.

[0028] The visual measuring instrument is also integrated with a controller and a computing power mainboard.

[0029] The visual automatic calibration system further includes an X-moving module and a Y-moving module, wherein the Y-moving module is vertically connected to the X-moving module, and the underwater stereo camera is mounted on the Y-moving module through a bracket, and the underwater stereo camera can move along the X and Y directions. The X-moving module and the Y-moving module are respectively connected to the controller.

[0030] The automatic visual calibration system further comprises a rotating platform, the underwater self-luminous calibration plate is connected to the rotating platform, and the rotating platform can control the underwater self-luminous calibration plate to rotate. The rotating platform is connected to a controller.

[0031] The control unit is a voltage control unit, and the power supply unit is a 12V adjustable DC power supply. The underwater self-luminous calibration board is waterproofed by the prior art and can work normally underwater.

[0032] Traditional calibration targets make the corners on the target easy to detect through material reflection, which shows good results in terrestrial environments. However, due to the influence of uneven illumination and water scattering, the limitations of traditional calibration targets in underwater multi-positioning and acquisition of a large number of corners have become apparent.

[0033] In order to ensure reliable detection of corner points on the calibration target plate in an underwater environment, the present invention designs an underwater self-luminous calibration plate, including a calibration plate body 10, on which a plurality of underwater LED units 20 are arranged in a matrix, and the light emission wavelength of the underwater LED units 20 is 530nm. The surface is designed to be flat to ensure consistent light intensity distribution.

[0034] There are 130 underwater LED units 20 in total, arranged in a matrix of 10 rows and 13 columns. The distance between adjacent underwater LED units 20 is 30 mm, and the size deviation is 0.01 mm. A spherical LED lamp 30 is added in the 3rd row and 2nd column to locate the position of the calibration plate.

[0035] The automatic visual calibration method of the present invention comprises the following steps:

[0036] Step S1, placing the underwater self-luminous calibration plate and the vision measuring instrument in water, taking the calibration target that is closest to the vision measuring instrument and can be clearly imaged as the initial calibration plane, and taking the first corner point in the lower left corner of the initial calibration plane as the origin of the world coordinate system.

[0037] Step S2: two underwater stereo cameras collect multiple sets of underwater self-luminous calibration plate image information from different angles.

[0038] In step S2, two underwater stereo cameras collect 80 stereo images from different left and right ranges to obtain a 10,400 sample data set. The data set is divided into a training set (70%), a validation set (20%) and a test set (10%). The training set is used for parameter training, the validation set is used for model adjustment, and the test set is used to evaluate model performance.

[0039] Step S3, using a neural network to express the mapping relationship between pixel coordinates and the 3D world plane.

[0040] In step S3, the neural network is a 3D coordinate regression network, and a fully connected neural network (FCNN) is used to regress the Z coordinate.

[0041] The fully connected neural network (FCNN) architecture includes an input layer, a hidden layer, and an output layer, and each layer is connected by weights.

[0042] The input layer includes 5 neuron nodes, including the image coordinates and corner point disparity captured by the stereo camera, and the output parameter is the corner point value in the world coordinate.

[0043] For X and Y coordinate regression, a clear mapping between the monocular camera image coordinates and the world coordinate plane is considered. First, the stereo camera is divided into two monocular cameras and two separate FCNNs are used to regress the X and Y coordinates. Finally, weights are assigned to the predicted pixels according to their distance from the principal points of the left and right images for fusion.

[0044] In step S3, the image data collected by the two underwater stereo cameras are fused. The stereo camera is conceptualized as a fusion system composed of monocular cameras. For X and Y coordinate regression, the mapping between the monocular camera pixel coordinates and the world coordinate plane is retained, and the predicted values ​​of the X and Y coordinates are obtained from the two underwater stereo cameras.

[0045] Step S4, extracting image features by extracting the center of underwater light spot.

[0046] In step S4, the underwater light spot is divided into a core area, an edge area, and a background area according to the number of pixels of each brightness level in the light spot image. The average grayscale value of the core area, the edge area, and the background area is calculated respectively, and the centroid is extracted by using the grayscale centroid method. The image coordinates and their respective grayscale values ​​are used as weights, and the center of the light spot is determined by calculating the weighted average.

[0047] Step S5, generating a calibration result based on all the image data after feature extraction.

[0048] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the visual automatic calibration method of the present invention are implemented.

[0049] Compared with the prior art, the visual automatic calibration system and method of the present invention are efficient and accurate. The data-driven calibration simplifies the camera calibration process into network learning. Compared with the traditional underwater calibration method, the calibration accuracy is high and the complexity is reduced by eliminating the requirement for understanding the imaging model. The designed underwater self-luminous calibration target system effectively reduces the influence of water scattering and is combined with the proposed underwater light spot extraction method to improve the calibration accuracy.

[0050] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any modifications, equivalent substitutions and improvements made within the concept of the present invention should be included in the patent protection scope of the present invention.

Claims

1. A visual automatic calibration system, characterized in that: It includes a visual measuring instrument, an underwater self-luminous calibration plate, a control unit and a power supply unit, wherein the underwater self-luminous calibration plate and the power supply unit are respectively connected to the control unit, the power supply unit is connected to the underwater self-luminous calibration plate, and the visual measuring instrument faces the underwater self-luminous calibration plate; The underwater self-luminous calibration plate comprises a calibration plate body, on which a plurality of underwater LED units are arranged in a matrix, wherein the light emission wavelength of the underwater LED units is 530 nm, the surface is designed to be flat, and the light intensity distribution is uniform.

2. The automatic visual calibration system, method and storage medium according to claim 1, characterized in that: There are a total of 130 underwater LED units, which are arranged in a matrix of 10 rows and 13 columns. The distance between adjacent underwater LED units is 30 mm. A spherical LED lamp is added in the 3rd row and 2nd column to locate the position of the calibration plate.

3. The automatic visual calibration system, method and storage medium according to claim 1, characterized in that: The visual measuring instrument includes two underwater stereo cameras, and the underwater stereo cameras adopt Daheng MER2-2000-6GM industrial cameras.

4. The automatic visual calibration system, method and storage medium according to claim 1, characterized in that: The visual automatic calibration system also includes an X-moving module and a Y-moving module, wherein the Y-moving module is vertically connected to the X-moving module, the underwater stereo camera is mounted on the Y-moving module through a bracket, and the underwater stereo camera can move along the X and Y directions; the X-moving module and the Y-moving module are respectively connected to the controller.

5. The automatic visual calibration system, method and storage medium according to claim 1, characterized in that: The automatic visual calibration system also includes a rotating platform, the underwater self-luminous calibration plate is connected to the rotating platform, and the rotating platform can control the rotation of the underwater self-luminous calibration plate; the rotating platform is connected to a controller.

6. A visual automatic calibration method, characterized in that: Applied to the visual automatic calibration system according to any one of claims 1 to 5, the visual automatic calibration method comprises the following steps: Step S1, placing the underwater self-luminous calibration plate and the vision measuring instrument in water, taking the calibration target that is closest to the vision measuring instrument and can be clearly imaged as the initial calibration plane, and taking the first corner point in the lower left corner of the initial calibration plane as the origin of the world coordinate system; Step S2, two underwater stereo cameras collect multiple sets of underwater self-luminous calibration plate image information from different angles; Step S3, using a neural network to express the mapping relationship between pixel coordinates and the 3D world plane; Step S4, extracting image features by using an underwater spot center extraction method; Step S5, generating a calibration result based on all the image data after feature extraction.

7. The automatic visual calibration method according to claim 6, characterized in that: In step S3, the image data collected by the two underwater stereo cameras are fused; for X and Y coordinate regression, the mapping between the monocular camera pixel coordinates and the world coordinate plane is retained, and the predicted values ​​of the X and Y coordinates are obtained from the two underwater stereo cameras.

8. The automatic visual calibration method according to claim 6, characterized in that: In step S4, the underwater light spot is divided into a core area, an edge area, and a background area according to the number of pixels of each brightness level in the light spot image; the average grayscale values ​​of the core area, the edge area, and the background area are calculated respectively, and the centroid is extracted by using the grayscale centroid method, and the image coordinates and their respective grayscale values ​​are used as weights to determine the center of the light spot by calculating the weighted average value.

9. The automatic visual calibration method according to claim 6, characterized in that: In step S3, the neural network is a 3D coordinate regression network, and a fully connected neural network (FCNN) is used to regress the Z coordinate; the fully connected neural network (FCNN) architecture includes an input layer, a hidden layer and an output layer, and each layer is connected by weights.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the visual automatic calibration method according to any one of claims 6 to 9 are implemented.

Images