L-rod camera calibration method and device based on visible light

Abstract

The invention relates to the technical field of optical motion capture, in particular to an L-rod camera calibration method and device based on visible light, and the method comprises the steps: collecting L-rod image data shot by a plurality of cameras at different visual angles, and obtaining the three-dimensional coordinates of an L-rod; selecting a region of interest by monitoring a mouse frame, and converting an RGB image of the ROI into a brightness space so as to extract a highlight point and obtain a two-dimensional coordinate of a small ball; implementing a small ball coding algorithm, coding according to four small balls, and realizing matching of two-dimensional image coordinates and three-dimensional coordinates; calculating an internal parameter matrix, and calculating external parameters by combining the position information of different small balls; calculating error verification calibration precision, and reconstructing a T rod for error test to determine the quality of a camera model; a brightness channel is adopted to screen a highlight area, the ball center precision of a small ball is improved, the light reflection influence is reduced, and the calibration precision is improved; a coding algorithm based on the positions of the four small balls is provided, accurate sorting is ensured, high matching of two-dimensional points and three-dimensional coordinates is achieved, and coding robustness is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of optical motion capture, and in particular to a visible light-based L-rod camera calibration method and device. Background Art

[0002] L-rod camera calibration is an important part of optical motion capture technology. Its main purpose is to obtain the internal and external parameters of the camera through multiple calibration points on the calibration rod (L-rod), so as to achieve accurate 3D reconstruction and motion capture of the object photographed by the camera. L-rod calibration is usually used in virtual reality, augmented reality, motion analysis, robot control and other fields.

[0003] In the traditional L-rod camera calibration method, two main technologies are usually used: active optical motion capture and passive optical motion capture. Active optical motion capture technology places light-emitting diodes at the specified position of the captured object in advance, uses these light-emitting diodes as marker points, and captures the motion trajectory of these marker points. For example, the calibration strategy proposed by Zhilai Education Technology belongs to active calibration technology. The company installs light-emitting diodes on the calibration ball to achieve the purpose of capturing the trajectory of the ball. However, the light-emitting diode has a certain degree of diffuse reflection from certain angles, which affects the calibration accuracy of the ball. In addition, this technology will rely on infrared receivers, which will inevitably cause the problem of high costs. The passive optical motion capture system mainly uses some reflective marker balls. The motion trajectory of these marker balls can be easily captured by the camera. For example, the calibration method proposed by qualisys and Metrics Technology belongs to passive optical motion capture technology, which achieves the purpose of tracking the trajectory of the ball by applying specific reflective materials on the ball. However, this method will increase the cost of the L-rod and also require a specific infrared receiver to receive the light reflected from the surface of the ball, and accurately locate the ball by estimating the depth of the returned light. However, the introduction of infrared equipment is bound to increase costs, thereby increasing the cost and complexity of the system.

[0004] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0005] The present invention provides an L-rod camera calibration method and device based on visible light, thereby effectively solving the problems in the background technology.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a L-rod camera calibration method based on visible light, comprising the following steps:

[0007] Collecting L-rod image data captured by multiple cameras at different viewing angles, and acquiring the three-dimensional coordinates of the L-rod based on the image data;

[0008] Selecting an area of ​​interest in the image data by monitoring the mouse frame technology, and converting the RGB image of the area of ​​interest into a brightness space to extract a highlight point and obtain the two-dimensional coordinates of the ball on the L rod;

[0009] According to the two-dimensional coordinates of the balls, a ball encoding algorithm is implemented to encode the four balls in a predetermined order so that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates;

[0010] Calculate the camera intrinsic parameters and determine the camera intrinsic parameter matrix according to the lens focal length, pixel size and resolution in the camera manual;

[0011] Calculating camera extrinsics, including rotation matrix and translation vector, based on the L-rod image data captured at different viewing angles;

[0012] The accuracy of camera calibration is verified by calculating the error value between the three-dimensional coordinates of the L rod and the actual three-dimensional coordinates;

[0013] Based on the camera intrinsic parameters and the camera extrinsic parameters, the length of the T-bar is reconstructed and an error test is performed with the actual physical length. The quality of the camera model is determined through the error test.

[0014] Further, a region of interest is selected in the image data by monitoring the mouse frame, and the RGB image of the region of interest is converted into a brightness space to extract highlight points and obtain the two-dimensional coordinates of the ball on the L rod; the steps include:

[0015] Selecting an area of ​​interest in the image data by monitoring the mouse frame technology, and converting the RGB image of the area of ​​interest into a color space and a brightness space to extract a highlight;

[0016] Calculating the brightness value L of the region of interest, and determining the ball category by the maximum value of the brightness value L;

[0017] The four balls are uniquely encoded according to a predetermined encoding rule to ensure that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates.

[0018] Furthermore, the ball encoding algorithm encodes the ball in a predetermined order from the long side to the short side according to a predetermined encoding rule, so as to ensure that the coordinates of the ball extracted from different viewing angles have a consistent arrangement order.

[0019] Furthermore, the implementation of the small ball coding algorithm comprises the following steps:

[0020] Initialize the positions of the four balls, marked as P1, P2, P3, and P4, and obtain the three-dimensional coordinates of the corresponding balls;

[0021] According to the distance relationship between the balls, the cosine value cosθ of the angle between the two balls is calculated to determine whether the two balls are on the same straight line;

[0022] If yes, then select the two closest balls and set them as the point O with the shortest distance, and determine the positions of the remaining balls based on the point O;

[0023] If not, calculate the spatial positions of the other balls according to the point O that is closest to them, and get the coordinates of the three balls P1, P2, and P3;

[0024] Perform conditional judgment on the three ball coordinates P1, P2, and P3 to determine whether the predetermined coordinate conditions are met;

[0025] If yes, the current ball position is determined to be a valid position, and the numbering sequence of the balls is updated;

[0026] If not, re-optimize the coordinate position based on the closest ball to ensure that the ball position meets the predetermined conditions;

[0027] Repeat the above steps until the positions of all balls meet the calibration requirements and the unique encoding process is completed.

[0028] Furthermore, the predetermined coordinate conditions include:

[0029] x min ≤P ix ≤x max ;

[0030] y min ≤P iy ≤y max ;

[0031] Among them, P ix is the x coordinate of the ball in a two-dimensional plane or three-dimensional space, P iy is the y coordinate of the ball in a two-dimensional plane or three-dimensional space, x min 、x max ,y min and max It is the preset spatial coordinate range.

[0032] Furthermore, the model of the internal parameter matrix includes:

[0033]

[0034] Where K is the camera intrinsic parameter matrix, f is the lens focal length, dx and dy are the pixel sizes, and H and W are the image resolutions.

[0035] Furthermore, the calculation of the camera extrinsic parameters adopts the PnP pose estimation algorithm, and the rotation matrix and translation vector of the camera are solved based on the two-dimensional image coordinates of multiple balls and their corresponding three-dimensional space coordinates, where:

[0036] The least square method is used for initial solution to obtain the preliminary rotation matrix and translation vector;

[0037] The PnP solution is optimized by combining the RANSAC algorithm to remove outliers.

[0038] Furthermore, the error value is an average reprojection error, which is obtained by calculating the average value of the Euclidean distance between the two-dimensional reprojection coordinates of all the balls and the actual two-dimensional coordinates.

[0039] Furthermore, the step of reconstructing the T-bar length includes:

[0040] Get the three-dimensional coordinates of the balls at both ends of the T-bar;

[0041] Calculate the Euclidean distance between the two balls as the reconstruction length;

[0042] The reconstructed length is compared with the real physical length to obtain the result of the error test.

[0043] The present invention also includes a multi-microgrid integrated energy system distributed optimization device, using the above method, including:

[0044] A data acquisition module, used to acquire L-rod image data captured by multiple cameras at different viewing angles, and obtain the three-dimensional coordinates of the L-rod based on the image data;

[0045] An image processing module is used to select an area of ​​interest in the image data by monitoring the mouse frame technology, and convert the RGB image of the area of ​​interest into a brightness space to extract a highlight point and obtain the two-dimensional coordinates of the ball on the L rod;

[0046] A ball encoding module, for implementing a ball encoding algorithm according to the two-dimensional coordinates of the balls, and encoding the four balls in a predetermined order so that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates;

[0047] The camera intrinsic parameter calculation module is used to calculate the camera intrinsic parameters according to the lens focal length, pixel size and resolution in the camera manual, and determine the camera intrinsic parameter matrix;

[0048] A camera extrinsic parameter calculation module, used to calculate the camera extrinsic parameters, including rotation matrix and translation vector, according to the L-rod image data taken at different viewing angles;

[0049] A verification module, used to verify the accuracy of camera calibration by calculating the error value between the reconstructed three-dimensional coordinates of the L-rod and the actual three-dimensional coordinates;

[0050] The testing module reconstructs the length of the T-bar based on the camera intrinsic parameters and the camera extrinsic parameters, and performs an error test with the actual physical length, and determines the quality of the camera model through the error test.

[0051] The beneficial effects of the present invention are:

[0052] It is proposed to convert the RGB image into the brightness channel to obtain a brightness map, and based on the brightness map, the highlight part is screened out to improve the accuracy of the center of the calibration ball, reducing the influence of external factors such as light reflection on the calibration accuracy, thereby improving the accuracy of the calibration result.

[0053] A four-ball position coding algorithm is proposed, which allows two-dimensional points to highly match corresponding three-dimensional coordinates. Through the precise sorting of the graph, the present invention has strong robustness in coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 It is a flowchart of the L-rod camera calibration method based on visible light;

[0056] Figure 2 This is a schematic diagram of the coding sequence of the L calibration rod;

[0057] Figure 3 A flowchart of the ball encoding algorithm;

[0058] Figure 4 Before and after comparison pictures for brightness screening;

[0059] Figure 5 Visualization diagram of the coding sequence for L-bar;

[0060] Figure 6 The reconstruction result for L rod;

[0061] Figure 7 Schematic diagram of the structure of the L-rod camera calibration device based on visible light. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0063] like Figures 1 to 6 As shown: A visible light based L-rod camera calibration method comprises the following steps:

[0064] S10: collecting L-rod image data captured by multiple cameras at different viewing angles, and obtaining the three-dimensional coordinates of the L-rod based on the image data;

[0065] S20: selecting a region of interest (ROI) in the image data by monitoring the mouse frame technology, and converting the RGB image of the region of interest into a brightness space to extract highlight points and obtain the two-dimensional coordinates of the ball on the L rod;

[0066] S30: implementing a ball encoding algorithm according to the two-dimensional coordinates of the balls, encoding the four balls in a predetermined order, so that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates;

[0067] S40: Calculate the camera intrinsic parameters according to the lens focal length, pixel size and resolution in the camera manual, and determine the camera intrinsic parameter matrix;

[0068] S50: Calculate camera extrinsic parameters, including rotation matrix and translation vector, according to L-rod image data captured at different viewing angles;

[0069] S60: Calculating an error value between the reconstructed three-dimensional coordinates of the L-rod and the actual three-dimensional coordinates to verify the accuracy of the camera calibration;

[0070] S70: Based on the camera intrinsic parameters and the camera extrinsic parameters, reconstruct the length of the T-bar and perform an error test with the actual physical length, and determine the quality of the camera model through the error test.

[0071] First, an innovative method is proposed to obtain the area of ​​interest by monitoring the mouse frame. Second, based on the area of ​​interest, it is innovatively converted into brightness space, aiming to obtain the coordinates of the highlight points of the balls and to accurately determine the coordinates of the balls. Third, based on the refined coordinates of the four balls, an innovative ball encoding sequence algorithm is proposed.

[0072] First, we innovatively proposed to convert the RGB image into the brightness channel to obtain a brightness map, and based on the brightness map, we screened out the highlight parts to improve the accuracy of calibrating the center of the ball; second, we innovatively proposed a position encoding algorithm based on four balls, so that the two-dimensional points can be highly matched with the corresponding three-dimensional coordinates.

[0073] This method uses visible light calibration, avoiding the use of infrared devices and light-emitting diodes commonly used in traditional calibration methods, and reducing equipment costs. Compared with systems that rely on infrared sensors, this method has lower hardware requirements and can use ordinary RGB cameras for calibration, further reducing the overall cost of the system. This method uses image data based on multiple cameras for calibration and performs precise positioning through simple RGB image processing. It does not require complex equipment or special environmental settings and has good applicability and flexibility. Since it can be operated in an ordinary visible light environment, this calibration method can be widely used in a variety of scenarios, whether indoors or outdoors, and can provide reliable calibration results.

[0074] By using the brightness space to extract the highlights in the image and obtain the 2D coordinates of the ball, the accuracy of the ball coordinates can be significantly improved. This processing method effectively reduces the impact of external factors such as light reflection on the calibration accuracy, thereby improving the accuracy of the calibration results. The ball encoding algorithm ensures the precise matching of the 2D image coordinates and the 3D coordinates, and enhances the accuracy of the 3D coordinate reconstruction.

[0075] By comparing the reconstructed L-rod 3D coordinates with the actual 3D coordinates, this method can effectively calculate the error and verify the accuracy. This helps to timely discover and correct possible errors during the calibration process and ensure the accuracy of the final result. Through continuous comparison and calibration, the method has high robustness and stability in practice and is suitable for calibration under multiple different perspectives and complex environments.

[0076] This method is compatible with existing optical capture systems and does not require additional complex hardware support, making it easy to quickly implement and promote its use. In addition, as a calibration method based on visible light, this technology can adapt to different types of cameras and equipment and has a wide range of applications, including industrial production, virtual reality, motion capture, robot vision and many other fields.

[0077] As a preferred embodiment of the above, in step S20, a region of interest (ROI) is selected in the image data by monitoring the mouse frame, and the RGB image of the region of interest is converted into a brightness space to extract highlight points and obtain the two-dimensional coordinates of the ball on the L rod; the steps include:

[0078] S21: selecting a region of interest (ROI) in the image data by monitoring the mouse frame technology, and converting the RGB image of the region of interest into a color space and a brightness space to extract a highlight;

[0079] S22: Calculate the brightness value L of the region of interest (ROI), and determine the ball category by the maximum value of the brightness value L;

[0080] S23: Uniquely encode the four balls according to a predetermined encoding rule to ensure that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates.

[0081] The mouse frame monitoring technology is used to accurately select the region of interest (ROI) to avoid interference from irrelevant backgrounds, thereby improving the accuracy of ball detection. The RGB conversion to brightness space is used to screen highlights to reduce color interference and make the recognition of balls more accurate. The calculation of the brightness value L is based on the highlights inside the ROI, not the entire image area. Therefore, even if the lighting conditions change, the brightness information of the ball can still be reliably extracted, improving the applicability of the calibration algorithm. The ball category is determined by the maximum brightness value L to ensure consistent classification of balls under different viewing angles, reduce misjudgment, and improve coding consistency. Select the ROI first and then perform color conversion to avoid unnecessary calculations for the entire image and improve computing efficiency. The ball classification is directly based on the maximum brightness value, without relying on complex morphological processing or other high-computation algorithms, making the processing speed faster. The ball classification is directly based on the maximum brightness value, without relying on complex morphological processing or other high-computation algorithms, making the processing speed faster. This encoding rule can remain stable under different camera viewing angles, ensure the matching consistency between the two-dimensional image and the three-dimensional structure, and improve the overall calibration accuracy.

[0082] In this embodiment, in step S30, Figure 2 As shown in the figure, the ball encoding algorithm encodes in a predetermined order from long side to short side according to a predetermined encoding rule to ensure that the ball coordinates extracted from different viewing angles have a consistent arrangement order. Through this encoding method, the matching relationship between the two-dimensional image coordinates and the three-dimensional ball coordinates is unique, thereby improving the accuracy and stability of camera calibration.

[0083] As a preference of the above embodiment, in step S30, Figure 3 As shown, the small ball encoding algorithm is implemented, and the steps include:

[0084] S31: Initialize the positions of four balls, marked as P1, P2, P3, and P4 respectively, and obtain the three-dimensional coordinates of the corresponding balls;

[0085] S32: Calculate the cosine value cosθ of the angle between the two balls according to the distance between them, and check whether the cosθ value < 7e-4 is satisfied, so as to determine whether the two balls are on the same straight line.

[0086] If yes, select the two closest balls and set them as point O with the shortest distance, and determine the positions of the remaining balls based on point O;

[0087] If not, calculate the spatial positions of the other balls according to the point O that is closest to them, and get the coordinates of the three balls P1, P2, and P3;

[0088] S33: Perform conditional judgment on the three ball coordinates P1, P2, and P3 to determine whether they meet the predetermined coordinate conditions;

[0089] If yes, the current ball position is determined to be a valid position, and the numbering sequence of the balls is updated;

[0090] If not, re-optimize the coordinate position based on the closest ball to ensure that the ball position meets the predetermined conditions;

[0091] S34: Repeat the above steps S33 to S33 until the positions of all balls meet the calibration requirements and the unique encoding process is completed.

[0092] The cosine value of the angle cosθ is used to calculate the relative position between the balls to ensure that the arrangement order of the balls remains consistent under different camera viewing angles, avoiding coding confusion caused by different shooting angles. By setting the reference point O, the relative position of the balls is calculated based on the distance to ensure that the order of the balls strictly matches the three-dimensional structure and improve the stability of the coding. By calculating the cosine value cosθ between the balls, it is possible to automatically determine whether the balls are collinear to avoid errors in ball coding due to errors. If the balls are collinear, the reference point is adjusted to ensure that the encoding process is not disturbed by irregular distribution, thereby improving the robustness of the algorithm. By re-optimizing based on the position of the nearest ball, incorrect classification is avoided and the accuracy of the ball position calculation is improved. According to the predetermined coordinate conditions, the position of the ball is adjusted through multiple iterations to ensure that the final calibrated ball coordinates are highly accurate. The closest point optimization strategy is adopted to adapt to different shooting angles, reduce errors, and further improve the calibration accuracy.

[0093] Wherein, in step S33, the predetermined coordinate conditions include:

[0094] x min ≤P ix ≤x max ;

[0095] y min ≤P iy ≤y max ;

[0096] Among them, P ix is the x coordinate of the ball in a two-dimensional plane or three-dimensional space, P iy is the y coordinate of the ball in a two-dimensional plane or three-dimensional space, x min 、x max ,y min and max It is a pre-set spatial coordinate range, ensuring that the position of the ball meets the calibration requirements and is within the valid spatial range.

[0097] By constraining the ball coordinates Pix and Piy in the preset spatial range [x min , x max ] and [y min ,y max ], effectively avoiding the coordinate deviation of the ball due to misidentification or noise interference, thereby ensuring that the position of the ball meets the calibration requirements. This method enables the two-dimensional image coordinates of the ball to accurately match the three-dimensional space coordinates, thereby improving the calibration accuracy of the L rod. min 、x max ,y min ,y max , ensuring that the ball detection does not exceed the valid area, preventing abnormal data from interfering with the calibration process. This restriction can prevent the ball from being mistakenly identified in the invalid area due to perspective distortion, misdetection or occlusion, and improve the stability and reliability of the calibration data.

[0098] In this embodiment, in step S40, the model of the internal parameter matrix includes:

[0099]

[0100] Where K is the camera intrinsic parameter matrix, f is the lens focal length, dx and dy are the pixel sizes, and H and W are the image resolutions.

[0101] The projection relationship from the camera's three-dimensional space to the two-dimensional image is established through the intrinsic parameter matrix K, so that the ball coordinates can be accurately mapped from the image plane to the three-dimensional space. The precise proportional relationship between the pixel coordinates and the physical size is calculated through the focal length f, pixel size dx, dy, to eliminate the camera's perspective distortion and improve the ball positioning accuracy.

[0102] As a preferred embodiment of the above embodiment, in step S50, the camera extrinsic parameters are calculated using a PnP (Perspective-n-Point) pose estimation algorithm, and the rotation matrix and translation vector of the camera are solved based on the two-dimensional image coordinates of multiple balls and their corresponding three-dimensional space coordinates, where:

[0103] The least square method is used for initial solution to obtain the preliminary rotation matrix R and translation vector T;

[0104] The RANSAC (Random Sampling Consensus) algorithm is used to optimize the PnP solution results to remove outliers and improve the robustness and accuracy of extrinsic parameter calculation.

[0105] In real scenes, the detection of small balls may be affected by noise interference, occlusion, illumination changes and other factors, resulting in mismatching of the 2D image coordinates and 3D space coordinates of some small balls. The RANSAC (Random Sampling Consistency) algorithm can effectively detect and remove outliers through random sampling + iterative optimization, ensuring the stability of extrinsic parameter calculation. Even if there is a certain proportion of mismatched points in the data, it can maintain a high calculation accuracy. The least squares method performs initial solution through global optimization and can obtain a smoother rotation matrix R and translation vector T, but it is easily disturbed by outliers. Combined with RANSAC for optimization, the optimal internal point set can be screened out for PnP calculation, so that the final rotation matrix R and translation vector T are more consistent with the real camera model and the accuracy of extrinsic parameter calculation is improved.

[0106] Among them, in step S60, the error value is the average reprojection error, which is obtained by calculating the average value of the Euclidean distance between the two-dimensional reprojection coordinates of all the balls and the actual two-dimensional coordinates; the overall accuracy of the entire L-rod calibration system is comprehensively evaluated to avoid the interference of local errors on the verification results. The Euclidean distance mean rather than the extreme value is used to effectively reduce the influence of single point noise (such as jitter in the edge extraction of the ball) on the error evaluation, and form a synergistic effect with the technical improvement of the brightness space conversion of the invention point to improve the coordinate accuracy; the average Euclidean distance directly reflects the pixel-level deviation of the calibrated model on the image plane, providing engineers with an intuitive reference for the error magnitude.

[0107] In this embodiment, in step S70, the step of reconstructing the length of the T-bar includes:

[0108] S71: Obtain the three-dimensional coordinates of the balls at both ends of the T-bar;

[0109] S72: Calculate the Euclidean distance between the two end balls as the reconstruction length;

[0110] S73: Compare the reconstructed length with the real physical length to obtain the result of the error test. Through the double verification of T-rod length reconstruction and L-rod reprojection error, a cross-check system is formed, which can effectively eliminate the accidental errors that may exist in a single calibration and ensure the reliability of the camera model; the abstract reprojection error (pixel level) is converted into a concrete physical length error (millimeter level), which significantly improves the interpretability and engineering practicality of the calibration results.

[0111] According to the invention point 1, it is innovatively proposed to convert the RGB image into the brightness channel to obtain a brightness map, and based on the brightness map, the highlight part is screened out to improve the accuracy of calibrating the center of the ball. The present invention provides a comparison diagram of the ball before and after the brightness screening diagram, such as Figure 4 shown.

[0112] According to the invention point 2, an algorithm based on the position coding of four small balls is innovatively proposed, so that the two-dimensional points can highly match the corresponding three-dimensional coordinates. The present invention will show the effect diagram of the L-bar sorting taken from various angles, such as Figure 5 As shown, through the precise sorting of the graph, the present invention has strong robustness in encoding.

[0113] The present invention obtains the camera parameter model of all cameras according to the above algorithm. In order to verify the accuracy of the camera parameter model, the present invention introduces a T-bar (0.505m) to perform a 3D reconstruction test to verify the length error calculated by the T-bar. At the same time, the present invention reconstructs and verifies the 3D structure of the L-bar according to pictures from various perspectives, such as Figure 6 The gray part represents the L-rod reconstructed by the camera model solved by the present invention, and the red part on the right represents the real three-dimensional coordinates of the L-rod. It can be seen that the length difference is the same.

[0114] The length error of the T-bar calculated for the three-dimensional reconstruction test verification is shown in Table 1. It can be seen that the error of the present invention in calculating the length of the multi-frame T-bar is also relatively low.

[0115] Table 1 T rod accuracy error

[0116]

[0117] The present invention also includes a distributed optimization device for a multi-microgrid integrated energy system, such as Figure 7 As shown, using the method as described above, including:

[0118] A data acquisition module, used to acquire L-rod image data captured by multiple cameras at different viewing angles, and obtain the three-dimensional coordinates of the L-rod based on the image data;

[0119] An image processing module is used to select a region of interest (ROI) in the image data by monitoring the mouse frame technology, and convert the RGB image of the region of interest into a brightness space to extract the highlight points and obtain the two-dimensional coordinates of the ball on the L rod;

[0120] A ball encoding module is used to implement a ball encoding algorithm according to the two-dimensional coordinates of the balls, and encode the four balls in a predetermined order so that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates;

[0121] The camera intrinsic parameter calculation module is used to calculate the camera intrinsic parameters according to the lens focal length, pixel size and resolution in the camera manual, and determine the camera intrinsic parameter matrix;

[0122] The camera extrinsic parameter calculation module is used to calculate the camera extrinsic parameters, including the rotation matrix and the translation vector, according to the L-rod image data taken at different viewing angles;

[0123] A verification module is used to verify the accuracy of camera calibration by calculating the error value between the reconstructed three-dimensional coordinates of the L-rod and the actual three-dimensional coordinates;

[0124] The testing module reconstructs the length of the T-bar based on the camera intrinsic parameters and the camera extrinsic parameters, and performs an error test with the actual physical length, and determines the quality of the camera model through the error test.

[0125] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. "Multiple" means two or more, unless otherwise clearly and specifically defined.

[0126] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0128] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0129] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0130] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0131] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0132] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A visible light based L-rod camera calibration method, characterized in that: The steps include: Collecting L-rod image data captured by multiple cameras at different viewing angles, and acquiring the three-dimensional coordinates of the L-rod based on the image data; Selecting an area of ​​interest in the image data by monitoring the mouse frame technology, and converting the RGB image of the area of ​​interest into a brightness space to extract a highlight point and obtain the two-dimensional coordinates of the ball on the L rod; According to the two-dimensional coordinates of the balls, a ball encoding algorithm is implemented to encode the four balls in a predetermined order so that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates; Calculate the camera intrinsic parameters and determine the camera intrinsic parameter matrix according to the lens focal length, pixel size and resolution in the camera manual; Calculating camera extrinsics, including rotation matrix and translation vector, based on the L-rod image data captured at different viewing angles; The accuracy of the camera calibration is verified by calculating the error value between the two-dimensional reprojection result of the three-dimensional coordinates of the L rod and the actual two-dimensional coordinates; Based on the camera intrinsic parameters and the camera extrinsic parameters, the length of the T-bar is reconstructed and an error test is performed with the actual physical length. The quality of the camera model is determined through the error test.

2. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The steps include: selecting an area of ​​interest in the image data by monitoring the mouse frame, and converting the RGB image of the area of ​​interest into a brightness space to extract a highlight point and obtain the two-dimensional coordinates of the ball on the L rod; Selecting an area of ​​interest in the image data by monitoring the mouse frame technology, and converting the RGB image of the area of ​​interest into a color space and a brightness space to extract a highlight; Calculating the brightness value L of the region of interest, and determining the ball category by the maximum value of the brightness value L; The four balls are uniquely encoded according to a predetermined encoding rule to ensure that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates.

3. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The ball encoding algorithm is based on a predetermined encoding rule and is encoded in a predetermined order from the long side to the short side to ensure that the coordinates of the balls extracted from different viewing angles have a consistent arrangement order.

4. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The implementation of the small ball coding algorithm comprises the following steps: Initialize the positions of the four balls, marked as P1, P2, P3, and P4, and obtain the three-dimensional coordinates of the corresponding balls; According to the distance relationship between the balls, the cosine value cosθ of the angle between the two balls is calculated to determine whether the two balls are on the same straight line; If yes, then select the two closest balls and set them as the point O with the shortest distance, and determine the positions of the remaining balls based on the point O; If not, calculate the spatial positions of the other balls according to the point O that is closest to them, and get the coordinates of the three balls P1, P2, and P3; Perform conditional judgment on the three ball coordinates P1, P2, and P3 to determine whether the predetermined coordinate conditions are met; If yes, the current ball position is determined to be a valid position, and the numbering sequence of the balls is updated; If not, re-optimize the coordinate position based on the closest ball to ensure that the ball position meets the predetermined conditions; Repeat the above steps until the positions of all balls meet the calibration requirements and the unique encoding process is completed.

5. The L-bar camera calibration method based on visible light according to claim 4, characterized in that: The predetermined coordinate conditions include: x min ≤P ix ≤x max ; and min ≤P iy ≤y max ; Among them, P ix is the x coordinate of the ball in a two-dimensional plane or three-dimensional space, P iy is the y coordinate of the ball in a two-dimensional plane or three-dimensional space, x min 、x max ,y min and max It is the preset spatial coordinate range.

6. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The model of the internal parameter matrix includes: Where K is the camera intrinsic parameter matrix, f is the lens focal length, dx and dy are the pixel sizes, and H and W are the image resolutions.

7. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The calculation of the camera extrinsic parameters adopts the PnP pose estimation algorithm, and solves the rotation matrix and translation vector of the camera based on the two-dimensional image coordinates of multiple balls and their corresponding three-dimensional space coordinates, where: The least square method is used for initial solution to obtain the preliminary rotation matrix and translation vector; The PnP solution is optimized by combining the RANSAC algorithm to remove outliers.

8. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The error value is an average reprojection error, which is obtained by calculating the average value of the Euclidean distances between the two-dimensional reprojection coordinates of all balls and the actual two-dimensional coordinates.

9. The L-bar camera calibration method based on visible light according to claim 1, characterized in that: The step of reconstructing the length of the T-bar comprises: Get the three-dimensional coordinates of the balls at both ends of the T-bar; Calculate the Euclidean distance between the two balls as the reconstruction length; The reconstructed length is compared with the real physical length to obtain the result of the error test.

10. A distributed optimization device for a multi-microgrid integrated energy system, characterized in that: Use of the method according to any one of claims 1 to 9, comprising: A data acquisition module, used to acquire L-rod image data captured by multiple cameras at different viewing angles, and obtain the three-dimensional coordinates of the L-rod based on the image data; An image processing module is used to select an area of ​​interest in the image data by monitoring the mouse frame technology, and convert the RGB image of the area of ​​interest into a brightness space to extract a highlight point and obtain the two-dimensional coordinates of the ball on the L rod; A ball encoding module, for implementing a ball encoding algorithm according to the two-dimensional coordinates of the balls, and encoding the four balls in a predetermined order so that the two-dimensional image coordinates accurately match the three-dimensional ball coordinates; The camera intrinsic parameter calculation module is used to calculate the camera intrinsic parameters according to the lens focal length, pixel size and resolution in the camera manual, and determine the camera intrinsic parameter matrix; A camera extrinsic parameter calculation module, used to calculate the camera extrinsic parameters, including rotation matrix and translation vector, according to the L-rod image data taken at different viewing angles; A verification module, used to verify the accuracy of camera calibration by calculating the error value between the two-dimensional reprojection result of the three-dimensional coordinates of the L rod and the actual two-dimensional coordinates; The testing module reconstructs the length of the T-bar based on the camera intrinsic parameters and the camera extrinsic parameters, and performs an error test with the actual physical length, and determines the quality of the camera model through the error test.

Images