Multi-view camera calibration method for stereo microscope and related device

Abstract

The application discloses a multi-view camera calibration method of a stereomicroscope and related devices, the method comprising: acquiring calibration board images of a calibration board in different poses, the calibration board being constructed by a plurality of regularly arranged grid lines; determining the extension direction of each grid line in the calibration board image; at a plurality of preset positions of the grid lines, acquiring a plurality of gray scale information of a plurality of target regions based on a cross-sectional direction perpendicular to the extension direction; determining the center line of the corresponding grid line according to the plurality of gray scale information; determining the intersection coordinates between any two center lines; and determining the intrinsic and extrinsic parameters of the multi-view camera according to the intersection coordinates. The application improves the accuracy of camera calibration in a microscopic scene.

Description

Technical Field

[0001] This application relates to the field of camera parameter calibration technology, and in particular to a multi-view camera calibration method and related apparatus for a stereo microscope. Background Technology

[0002] Image distortion is a widespread aberration in cameras that causes image distortion, leading to significant errors in image-based applications such as topography measurement and 3D modeling. Therefore, camera calibration, by solving for the camera's intrinsic and extrinsic parameters, eliminating distortion, and obtaining accurate camera parameters, is a core step in computer vision and image processing. Through camera calibration, distortion parameters can be precisely calculated and corrected, thereby restoring the true geometry of the image.

[0003] Currently, the most commonly used multi-view camera calibration method is the Zhang calibration method. It uses a multi-view camera to capture images of the distribution of each corner point in the chessboard calibration board under different poses, and compares the distribution with the theoretical spatial relationship between each corner point and the actual size of the chessboard to obtain the intrinsic and extrinsic parameters of the multi-view camera.

[0004] Zhang's calibration method requires a camera that approximates an ideal pinhole imaging model, meaning all corner points must be within the depth of focus for clear imaging. Therefore, it is highly sensitive to defocusing of the checkerboard calibration plate pattern recorded by the camera. Due to the shallow depth of field in microscopic scenes, there is a significant probability that the checkerboard calibration plate will tilt during calibration image capture under the microscope, causing corner points to exceed the depth of focus and become blurred into a hazy spot. This blurring significantly affects the accuracy of corner point detection, potentially leading to detection failure and impacting the accuracy of camera parameter calculations.

[0005] Meanwhile, this method requires that each calibration board image include all corner points of the checkerboard calibration board, but in the small field of view of a microscopic scene, it may be impossible to record all corner points, thus leading to calibration failure. Summary of the Invention

[0006] This application provides a method and apparatus for calibrating a multi-view camera in a stereomicroscope to improve the accuracy of camera calibration in microscopic scenarios.

[0007] In a first aspect, embodiments of this application provide a method for calibrating a multi-view camera on a stereo microscope, including:

[0008] Images of the calibration board are acquired when the calibration board is in different poses. The calibration board is constructed by multiple regularly arranged grid lines.

[0009] Determine the extension direction of each grid line in the calibration plate image;

[0010] At multiple preset positions of the grid lines, multiple grayscale information of multiple target regions is obtained based on the cross-sectional direction perpendicular to the extension direction;

[0011] The center line of the corresponding grid line is determined based on the multiple grayscale information;

[0012] Determine the coordinates of the intersection point between any two of the centerlines;

[0013] The intrinsic and extrinsic parameters of the multi-view camera are determined based on the coordinates of the intersection point.

[0014] The step of determining the center line of the corresponding grid line based on the plurality of grayscale information includes:

[0015] Multiple grayscale information values ​​are fitted to obtain multiple target fitting curves;

[0016] Determine multiple fitting midpoints of multiple target fitted curves;

[0017] The centerline is obtained by fitting multiple midpoints.

[0018] The process of fitting multiple grayscale information to obtain multiple target fitting curves includes:

[0019] Determine the target angle formed by the grid lines and the bottom edge of the calibration board image;

[0020] Obtain the width of the grid lines;

[0021] A first function is constructed based on the target angle, the width, and the initial control parameters;

[0022] The grayscale information is fitted according to the first function to obtain an initial fitting curve;

[0023] Determine the fitting effect of the initial fitted curve;

[0024] The initial control parameters are optimized based on the fitting effect to obtain the target control parameters;

[0025] The first function is modified according to the target control parameters to obtain the second function;

[0026] The second function is used to fit multiple grayscale information values ​​to obtain multiple target fitting curves.

[0027] The step of constructing the first function based on the target angle, the width, and the initial adjustment parameters includes:

[0028] Based on the target angle and the initial control parameters, a third function is constructed, which is used to characterize the distance between a pixel in a single target region and the center line of the corresponding grid line;

[0029] Determine the ratio of the third function to the width;

[0030] The first function is constructed based on the ratio.

[0031] The step of determining the intrinsic and extrinsic parameters of the multi-view camera based on the intersection point coordinates includes:

[0032] Based on the intersection coordinates and the preset calibration algorithm, determine the first intrinsic parameter and the first extrinsic parameter of each camera in the multi-view camera;

[0033] Acquire the standard camera from the multi-view cameras;

[0034] Based on the first intrinsic parameters of the standard camera and the first intrinsic parameters of the other cameras, the second extrinsic parameters of the other cameras relative to the standard camera are determined, wherein the other cameras are cameras other than the standard camera among the multi-view cameras;

[0035] Based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, determine the third extrinsic parameter of the other cameras;

[0036] The intrinsic and extrinsic parameters of the multi-camera are obtained based on the first intrinsic parameter of the standard camera, the first extrinsic parameter of the standard camera, the first intrinsic parameter of the other cameras, and the third extrinsic parameter of the other cameras.

[0037] The step of determining the third extrinsic parameter of the other cameras based on the second extrinsic parameter and the first extrinsic parameter of the standard camera includes:

[0038] Based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, determine the fourth extrinsic parameter of the other cameras;

[0039] The third extrinsic parameter is determined based on the first extrinsic parameter and the fourth extrinsic parameter of the other cameras.

[0040] The step of determining multiple fitting midpoints of the multiple target fitting curves includes:

[0041] Determine multiple height medians of the multiple target fitted curves;

[0042] Among the multiple target fitting curves, multiple width pairs corresponding to the multiple median heights are determined;

[0043] The plurality of fitted midpoints are determined based on the plurality of width pairs.

[0044] Secondly, embodiments of this application provide a multi-view camera calibration device for a stereo microscope, comprising:

[0045] The first acquisition unit is used to acquire images of the calibration board when the calibration board is in different poses, wherein the calibration board is constructed by multiple regularly arranged grid lines;

[0046] The first determining unit is used to determine the extension direction of each grid line in the calibration plate image;

[0047] The second acquisition unit is used to acquire multiple grayscale information of multiple target areas at multiple preset positions of the grid line, based on a cross-sectional direction perpendicular to the extension direction;

[0048] The second determining unit is used to determine the center line of the corresponding grid line based on the plurality of grayscale information;

[0049] The third determining unit is used to determine the coordinates of the intersection point between any two of the center lines;

[0050] The fourth determining unit is used to determine the intrinsic and extrinsic parameters of the multi-view camera based on the coordinates of the intersection point.

[0051] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and executable program code stored in the memory and executable on the processor, wherein the processor executes the executable program code and performs the steps of the method described in the first aspect.

[0052] Fourthly, embodiments of this application provide a computer-readable storage medium storing executable program code, the executable program code including execution instructions for performing the steps of the method as described in the first aspect.

[0053] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0054] As can be seen, in this embodiment of the application, firstly, calibration board images are acquired when the calibration board is in different poses. The calibration board is constructed by multiple regularly arranged grid lines. The extension direction of each grid line in the calibration board image is determined. At multiple preset positions of the grid lines, multiple grayscale information of multiple target areas is acquired based on the cross-sectional direction perpendicular to the extension direction. The center line of the corresponding grid line is determined according to the multiple grayscale information. The coordinates of the intersection point between any two of the center lines are determined. The intrinsic and extrinsic parameters of the multi-view camera are determined according to the intersection point coordinates.

[0055] This application collects grayscale information in the cross-sectional direction perpendicular to the extension direction of the grid lines at different locations along the grid lines. By collecting multiple grayscale information, the center line of each grid line is determined, and the coordinates of the intersection of any two center lines are calculated. The coordinates of the intersection of the center lines are used to replace the corner points of the checkerboard, thereby solving the defect of corner point detection failure caused by defocusing in microscopic scenes. At the same time, even if the calibration board image does not include all intersection points, this application can determine the positions of all intersection points based on multiple center lines, thereby completing the calibration of multi-view cameras and improving the accuracy of camera calibration in microscopic scenes. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This application provides a system architecture diagram of a camera calibration system according to an embodiment.

[0058] Figure 2 This is a schematic flowchart of a multi-view camera calibration method for a stereo microscope provided in an embodiment of this application;

[0059] Figure 3 This is a schematic diagram of a grid calibration plate provided in an embodiment of this application;

[0060] Figure 4 This is a schematic diagram of a target fitting curve provided in an embodiment of this application;

[0061] Figure 5 This is a flowchart illustrating another method for calibrating a multi-view camera in a stereo microscope, as provided in an embodiment of this application.

[0062] Figure 6 This is a block diagram of the functional units of a multi-view camera calibration device for a stereo microscope provided in an embodiment of this application;

[0063] Figure 7 This is a block diagram of the functional units of another stereo microscope multi-view camera calibration device provided in the embodiments of this application;

[0064] Figure 8 This is a schematic diagram of the structure of an electronic device proposed in an embodiment of this application. Detailed Implementation

[0065] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0066] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0067] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] Image distortion is a widespread aberration in cameras that causes image distortion, leading to significant errors in image-based applications such as topography measurement and 3D modeling. Therefore, camera calibration, by solving for the camera's intrinsic and extrinsic parameters, eliminating distortion, and obtaining accurate camera parameters, is a core step in computer vision and image processing. Through camera calibration, distortion parameters can be precisely calculated and corrected, thereby restoring the true geometry of the image.

[0069] Currently, the most commonly used multi-view camera calibration method is the Zhang calibration method. It uses a multi-view camera to capture images of the distribution of each corner point in the chessboard calibration board under different poses, and compares the distribution with the theoretical spatial relationship between each corner point and the actual size of the chessboard to obtain the intrinsic and extrinsic parameters of the multi-view camera.

[0070] Zhang's calibration method requires a camera that approximates an ideal pinhole imaging model, meaning all corner points must be within the depth of focus for clear imaging. Therefore, it is highly sensitive to defocusing of the checkerboard calibration plate pattern recorded by the camera. Due to the shallow depth of field in microscopic scenes, there is a significant probability that the checkerboard calibration plate will tilt during calibration image capture under the microscope, causing corner points to exceed the depth of focus and become blurred into a hazy spot. This blurring significantly affects the accuracy of corner point detection, potentially leading to detection failure and impacting the accuracy of camera parameter calculations.

[0071] Meanwhile, this method requires that each calibration board image include all corner points of the checkerboard calibration board, but in the small field of view of a microscopic scene, it may be impossible to record all corner points, thus leading to calibration failure.

[0072] To address the aforementioned issues, this application provides a method and apparatus for calibrating a multi-view camera for a stereo microscope. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0073] Please see Figure 1 , Figure 1 This is a system architecture diagram of a camera calibration system provided in an embodiment of this application. Figure 1 As shown, the camera calibration system 100 includes an image acquisition module 101, a grayscale information acquisition module 102, a fitting module 103, and a multi-view camera calibration module 104. The image acquisition module 101, the grayscale information acquisition module 102, the fitting module 103, and the multi-view camera calibration module 104 are interconnected in pairs.

[0074] The image acquisition module 101 is used to acquire images of the grid calibration plate in different poses using a multi-view camera of a microscope.

[0075] Specifically, the camera is fixed, and the position and rotation angle of the grid calibration plate are moved to change the spatial state of the grid calibration plate relative to the microscope multi-view camera. After each movement, the multi-view camera takes a set of calibration plate images. For example, 10-20 sets of images in different poses can be acquired.

[0076] The grayscale information acquisition module 102 is used to divide the grid lines in the acquired calibration board image. Based on each division position, the grayscale information of the target area is acquired in the cross-sectional direction perpendicular to the direction of the grid lines, so as to obtain the grayscale information of the target area corresponding to each division position.

[0077] The fitting module 103 is used to perform gray-level distribution fitting operations on multiple gray-level information of multiple target regions corresponding to each grid line to obtain the fitting result of each target region; and to determine the fitting midpoint of each fitting result, and then perform line fitting on each fitting midpoint to obtain the center line of each grid line.

[0078] The multi-view camera calibration module 104 is used to calculate the intersection of the center lines of any two grid lines to obtain the positions of all corner points in the calibration board image. By repeatedly performing grayscale information acquisition and fitting operations, the corner point positions of each calibration board image are obtained. Then, based on the corner point positions of each calibration board image, the intrinsic and extrinsic parameters of the multi-view camera are determined through a preset calibration method, thereby achieving the calibration of the multi-view camera.

[0079] Based on this, please refer to Figure 2 , Figure 2 This is a schematic flowchart of a multi-view camera calibration method for a stereo microscope provided in an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps:

[0080] S210, acquire calibration board images when the calibration board is in different poses.

[0081] The calibration board is constructed using a grid of multiple regular columns.

[0082] The process involves placing the grid calibration plate within the microscope's field of view and capturing images of the calibration plate using the microscope's multi-view camera. Synchronous triggering ensures that the multi-view camera acquires images of the calibration plate at the same time, thereby guaranteeing the consistency of the calibration plate's pose from different camera perspectives.

[0083] This involves acquiring clear images of the calibration board under different combinations of translation and rotation poses, obtaining multiple calibration board images, and then calibrating the camera based on these multiple calibration board images.

[0084] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a grid calibration plate provided in an embodiment of this application, as shown below. Figure 3 As shown, the calibration board is constructed using multiple regularly arranged grid lines, including multiple grids, each of uniform size and with a regular layout.

[0085] For example, the background of the grid calibration plate is black and the grid lines are white.

[0086] The grid formed by the grid lines can be a parallelogram, a rectangle, or a square.

[0087] In one possible embodiment, the grid is preferably rectangular in order to distinguish the two orientations of the grid lines.

[0088] For example, the calibration board can also be a white background with black grid lines. If a combination of white and black grid lines is chosen, then a grayscale inversion operation needs to be performed on the original image in the very first step of image processing to convert it into an image with a black background and white grid lines.

[0089] Among them, the acquired calibration board images may contain particle noise, which mainly comes from sensor thermal noise, light interference or circuit noise, etc. It manifests as small spots or particles of varying brightness randomly distributed on the image background or solid color area. Therefore, it is necessary to perform noise reduction processing on the calibration board images to remove particle noise and reduce the error caused by subsequent grid line fitting.

[0090] One method is to perform noise reduction by mean filtering, which takes the average value of all pixels in the neighborhood of a pixel as the current pixel value, such as taking the average of 9 pixels in a 3×3 neighborhood.

[0091] One of the methods is to perform noise reduction by Gaussian filtering, which uses a Gaussian function to weight the neighboring pixels, with high weight for the center pixels and low weight for the edge pixels.

[0092] Among them, noise reduction processing can also be performed by median filtering, which sorts all pixel values ​​in the pixel neighborhood and takes the median value as the current pixel value, such as taking the 5th largest pixel value in a 3×3 neighborhood.

[0093] In this embodiment, median filtering is preferably used to perform denoising processing on the calibration board image.

[0094] Specifically, the grain noise in the calibration board image is mostly isolated bright / dark points. The pixel values ​​of this type of noise are extremely different from the surrounding normal pixels. Median filtering can directly replace these extreme noise points with normal pixel values ​​in the neighborhood by sorting and taking the median value.

[0095] For example, if an overly bright noise point appears in a black grid, its 3×3 neighborhood includes 8 normal dark pixels and 1 bright noise point. After sorting, the middle value is the dark pixel, and the noise point is replaced, thus achieving targeted removal of grainy noise.

[0096] The grid calibration board is characterized by distinct black and white edges and sharp corners. The pixel values ​​of these two types of features exhibit abrupt changes in the neighborhood, such as a brightness jump from black to white. Median filtering can preserve clear edges and corners, providing high-quality image data for subsequent grid line fitting and camera calibration, and ultimately reducing calibration errors.

[0097] S220, determine the extension direction of each grid line in the calibration plate image.

[0098] In one possible embodiment, the target spatial spectrum of the calibration board image is first determined; the extension direction of each grid line in the calibration board image is determined based on the target spatial spectrum.

[0099] Specifically, a Fourier transform operation is performed on the denoised calibration board image to convert the image from the spatial domain to the frequency domain, and then the generated target spatial spectrum is analyzed to obtain the orientation of the grid lines.

[0100] Specifically, the core feature of the grid calibration plate is the periodic repeating black and white stripes. Through Fourier transform, the periodic gray-scale changes in the spatial domain are converted into discrete frequency components in the frequency domain, which are represented as bright spots in the spatial spectrum, i.e. frequency peaks.

[0101] In the spatial domain, the direction of the grid lines is the same as the angle of the bright spot relative to the origin in the frequency domain.

[0102] S230, at multiple preset positions of the grid line, based on the cross-sectional direction perpendicular to the extension direction, multiple grayscale information of multiple target areas are obtained.

[0103] Among them, multiple preset positions of the grid lines are determined based on the length or width of the grid in the calibration plate.

[0104] Specifically, the first angle formed by the extension direction of the grid line and the bottom edge of the calibration plate image is calculated, and the second angle formed by the extension direction of the grid line and the side edge of the calibration plate image is calculated. If the first angle is less than the second angle, the preset position is determined according to the length of the grid in the calibration plate; if the first angle is greater than the second angle, the preset position is determined according to the width of the grid in the calibration plate.

[0105] In one possible embodiment, the interval between two adjacent preset positions is the same.

[0106] In one possible embodiment, the interval between two adjacent preset positions is an integer multiple of the length or width of the grid.

[0107] Specifically, at each preset position of the grid line, the grayscale information of the target area in the cross-sectional direction is obtained according to the extension direction of the grid line.

[0108] Specifically, the target area needs to ensure that background grayscale values ​​are captured both before entering and after leaving the grid lines. The distance from the top of the target area to the corresponding preset position is equal to the distance from the bottom of the target area to the corresponding preset position.

[0109] The grayscale information includes the grayscale values ​​of all pixels in the target area.

[0110] S240, determine the center line of the corresponding grid line based on the multiple grayscale information.

[0111] In one possible embodiment, determining the center line of the corresponding grid line based on the plurality of grayscale information includes: fitting the plurality of grayscale information to obtain a plurality of target fitting curves; determining a plurality of fitting midpoints of the plurality of target fitting curves; and fitting the plurality of fitting midpoints to obtain the center line.

[0112] In the calibration plate image, there is a grayscale difference between the grid lines and the background. For example, the grayscale value of a white grid line on a black background will be significantly higher than that of the background. If a straight line is taken along the cross-sectional direction of the grid line, the grayscale value on this line will show a symmetrical distribution with the background being lower and the grid line being higher.

[0113] The acquired grayscale information is a discrete set of pixels, which cannot reflect the continuous gradual change of grayscale distribution. Therefore, it is necessary to use fitting to transform the discrete, noisy grayscale values ​​on each sampling line into a continuous, smooth curve, thereby eliminating the interference of image noise on grayscale distribution and preserving the grayscale symmetry characteristics of the grid lines.

[0114] In one possible embodiment, fitting multiple grayscale information to obtain multiple target fitting curves includes: determining the target angle formed by the grid lines and the bottom edge of the calibration board image; obtaining the width of the grid lines; constructing a first function based on the target angle, the width, and initial adjustment parameters; fitting the grayscale information according to the first function to obtain an initial fitting curve; determining the fitting effect of the initial fitting curve; optimizing the initial adjustment parameters based on the fitting effect to obtain target adjustment parameters; correcting the first function according to the target adjustment parameters to obtain a second function; and fitting multiple grayscale information according to the second function to obtain multiple target fitting curves.

[0115] The target angle θ is determined based on the direction of the grid lines.

[0116] Specifically, for grid lines that extend nearly horizontally, θ is the target angle formed by the extension direction and the bottom edge of the calibration board image; for grid lines that extend nearly vertically, θ is the target angle formed by the extension direction and the side edge of the calibration board image.

[0117] Specifically, based on the target angle, line width, and initial control parameters, the grayscale information of the grid line cross-section is fitted with a Butworth function to obtain the grayscale distribution of each grid line cross-section, i.e., the target fitting curve.

[0118] In one possible embodiment, constructing the first function based on the target angle, the width, and the initial control parameters includes: constructing a third function based on the target angle and the initial control parameters, wherein the first function is used to characterize the distance between a pixel in the image patch and the center line of the grid line to which the image patch belongs; determining the ratio of the third function to the width; and constructing the first function based on the ratio.

[0119] The expression for the third function is:

[0120] ,

[0121] in, Characterizing pixels The third function of distance to the centerline. These are the initial control parameters.

[0122] Among them, when It is a formula that describes the center line of the grid lines in an image.

[0123] The expression for the first function is:

[0124]

[0125] Where b is the width of the grid line, This is the first function.

[0126] Among them, initial control parameters are set for the third function. , The initial value can be chosen based on experience or randomly.

[0127] Among them, the pixel coordinates of the sampling line Substituting into the third function, we get After that Substitute the values ​​into the first function to calculate a set of fitted gray values. Connect the fitted gray values ​​sequentially to obtain the initial fitted curve.

[0128] Among them, the fitting parameter set is set. ,vector When performing fit analysis, a cost function is set. Used to characterize the true grayscale value With the output of the fitted model The sum of squared errors between them is expressed as:

[0129]

[0130] in, The smaller the value, the smaller the difference between the initial fitted curve and the true gray-level distribution, and the better the fitting effect; conversely, the larger the value, the worse the fit.

[0131] Through multiple iterations and continuous updates ,let Minimize as much as possible to achieve optimal results. ,from Obtain the optimal In order to obtain the target control parameters.

[0132] Specifically, The gradient descent process is represented as:

[0133] ,

[0134] in, For the descent speed, Let be the parameter after the k-th iteration, and be the successive approximation value of the target control parameter. These are the parameters for the (k-1)th iteration; For the cost function SSR with respect to parameters The gradient reflects the direction of the fastest change in error, and the gradient descent method adjusts the parameters along the direction of the fastest decrease in error.

[0135] In this process, the target control parameters obtained through gradient descent are substituted into the first function to obtain a new function, namely the second function.

[0136] Specifically, the optimized second function is used to fit each grayscale information of each target region to obtain the target fitting curve for each target region.

[0137] In one possible embodiment, please refer to Figure 4 , Figure 4 This is a schematic diagram of a target fitting curve provided in an embodiment of this application, such as... Figure 4 As shown, the target fitting curve is the response curve of the Butterworth filter.

[0138] In this model, the vertical axis y corresponds to the grayscale value of the grid line cross-section, and the horizontal axis x corresponds to the distance from the pixel to the center line of the grid line. The closer x is to 0, the higher the grayscale value.

[0139] The target fitting curve exhibits a unimodal shape, reaching its peak near x=0, gradually decreasing towards both sides, and eventually approaching 0. The curve is roughly symmetrical about x=0, and the gray-level distribution in this target region has central symmetry.

[0140] In one possible embodiment, determining multiple fitting midpoints of the plurality of target fitting curves includes: determining multiple height medians of the plurality of target fitting curves; determining multiple width pairs corresponding to the multiple height medians in the plurality of target fitting curves; and determining the plurality of fitting midpoints based on the multiple width pairs.

[0141] Specifically, the maximum and minimum values ​​in the target fitted curve are obtained, the difference between the maximum and minimum values ​​is calculated, and then the median of the difference is calculated to obtain the height median.

[0142] Specifically, the two x-values ​​corresponding to the median height are calculated, and then the mean of these two x-values ​​is calculated to obtain the fitted midpoint.

[0143] In this process, after calculating the midpoint of the fitting curve for each target, line fitting is performed on multiple fitting centers to obtain the center line corresponding to each grid line.

[0144] Among them, the least squares method can be used to perform line fitting for multiple fitting centers. The least squares method is used to solve the model parameters so that the overall deviation between the fitted center line and all discrete fitting centers is minimized.

[0145] As can be seen, in this embodiment, the feature point localization accuracy is improved from the pixel level to the sub-pixel level by single-region grayscale fitting. At the same time, the grayscale distribution of a single target region may be affected by local noise, causing feature point offset. Therefore, this embodiment fits the center line by feature points of multiple target regions and cancels the local deviation by least squares method, thereby improving the center line localization accuracy. This provides high-quality feature input for subsequent camera calibration tasks and improves the accuracy and reliability of camera calibration.

[0146] S250, determine the coordinates of the intersection point between any two of the center lines.

[0147] Specifically, the mathematical expressions for the centerlines of any two grid lines are taken, and their intersection points are calculated. By repeating the above steps, the coordinates of all grid line intersection points can be obtained, which are all corner points in the calibration board image.

[0148] S260, determine the intrinsic and extrinsic parameters of the multi-view camera based on the coordinates of the intersection point.

[0149] In one possible embodiment, determining the intrinsic and extrinsic parameters of the multi-camera system based on the intersection coordinates includes: determining the first intrinsic and first extrinsic parameters of each camera in the multi-camera system based on the intersection coordinates and a preset calibration algorithm; obtaining the standard camera in the multi-camera system; determining the second extrinsic parameters of other cameras relative to the standard camera based on the first intrinsic parameters of the standard camera and the first intrinsic parameters of the other cameras, wherein the other cameras are cameras in the multi-camera system other than the standard camera; determining the third extrinsic parameters of the other cameras based on the second extrinsic parameters and the first extrinsic parameters of the standard camera; and obtaining the intrinsic and extrinsic parameters of the multi-camera system based on the first intrinsic parameters of the standard camera, the first extrinsic parameters of the standard camera, the first intrinsic parameters of the other cameras, and the third extrinsic parameters of the other cameras.

[0150] After obtaining the coordinates of all grid line intersections, the first intrinsic and first extrinsic parameters of each camera are calculated based on the calibration board image corresponding to each camera, according to the preset calibration method.

[0151] Specifically, first, the origin of the world coordinate system is defined. For example, the first corner point of the top left corner of the calibration plate grid is selected as the world origin (X=0, Y=0, Z=0). Then, the coordinates of each corner point are calculated according to the actual physical size of the calibration plate. If the horizontal spacing between adjacent corner points of the grid is dx and the vertical spacing is dy, then for the corner point in the i-th row and j-th column, its world coordinates can be directly determined as X=j×dx, Y=i×dy, Z=0.

[0152] In this process, it is necessary to ensure that the Z coordinate of all corner points is always 0, and that the distance between adjacent corner points is consistent with the actual measurement value, so as to avoid coordinate deviation due to grid counting errors.

[0153] This involves obtaining the coordinates of the corner points in the image coordinate system.

[0154] Specifically, the mapping relationship between world coordinates and image coordinates is determined based on the corner coordinates in the world coordinate system and the corner coordinates in the image coordinate system. This mapping relationship can be described by the homography matrix H, and it satisfies the homogeneous coordinate equation.

[0155] The homography matrix H is a 3×3 matrix used to project 3D points of the calibration plate plane in the world coordinate system onto 2D points of the image.

[0156] Based on the above mapping relationship, each pair of world 3D points and image 2D points can establish two independent linear equations. Therefore, at least four non-collinear matching pairs are required to solve for the eight independent parameters.

[0157] The least squares method is used to solve the above equations. First, the homogeneous equations are transformed into a system of linear equations. Then, the coefficient matrix and observation vector are constructed. Finally, the optimal solution of the H matrix is ​​obtained.

[0158] Each calibration board image corresponds to an independent homography matrix H. Due to the different relative positions of the calibration board plane and the camera under different postures, the projection relationship is also different.

[0159] Among them, the camera intrinsic parameters are a set of parameters describing the camera's own optical characteristics, including the camera's effective focal length in the u and v axis directions; it also includes the coordinates of the principal point of the image, that is, the coordinates of the intersection of the camera's optical axis and the image plane, which should ideally be located at the center of the image, but may deviate from the center due to assembly errors; it also includes distortion coefficients, which are divided into radial distortion (k1, k2, k3) and tangential distortion (p1, p2), which need to be solved separately in the subsequent intrinsic parameter optimization.

[0160] Specifically, in the initial stage of solving the intrinsic parameters, the distortion coefficient can be temporarily ignored, or it can be assumed that the camera is an ideal camera with no distortion.

[0161] These parameters are integrated into an intrinsic parameter matrix K.

[0162] The homography matrix H is constructed based on the intrinsic parameter matrix K and the extrinsic parameters (rotation matrix R and translation vector t), i.e., H = K × [R1R2 t], where R1 and R2 are the first two columns of the rotation matrix R. Based on the above relationship, the constraint equations corresponding to each intrinsic parameter can be derived, and finally, the preliminary intrinsic parameter matrix, i.e., the first intrinsic parameter of the camera, can be obtained by solving the least squares method.

[0163] The extrinsic parameters are used to describe the relative position and attitude between the world coordinate system and the camera coordinate system. They consist of two parts: a rotation matrix R and a translation vector t. The rotation matrix R is a 3×3 orthogonal matrix used to describe the attitude of the camera coordinate system relative to the world coordinate system. The translation vector t is a 3×1 vector used to describe the position of the origin of the camera coordinate system in the world coordinate system.

[0164] Each set of extrinsic parameters corresponds to one calibration board image, that is, M calibration board images correspond to M sets of extrinsic parameters.

[0165] After solving the intrinsic parameter matrix, a calibration plate image parallel to the camera is selected. The extrinsic parameters can be directly obtained by performing matrix decomposition on the homography matrix H of the image, splitting H into the product of the intrinsic parameter matrix, rotation matrix, and translation vector, and then calculating the extrinsic parameters corresponding to the calibration plate image to obtain the rotation matrix R and the translation vector t.

[0166] In this context, the standard camera is the camera that is preset as the coordinate reference among the multiple cameras. The extrinsic parameters of all other cameras can be based on the parameters of the standard camera to achieve relative attitude consistency among the multiple cameras.

[0167] Specifically, the first extrinsic parameter of the standard camera is used to convert the world coordinates of the corner points to coordinates in the standard camera coordinate system, and the image coordinates of the corner points in other cameras are obtained. Taking 3D points in the standard camera coordinate system and 2D image points in other cameras as input, combined with the intrinsic parameters of other cameras, and based on a preset algorithm, the reprojection error of 3D points projected onto 2D images is minimized through the correspondence between 3D points and 2D points, and the rotation matrix and translation vector of the camera, i.e., the second extrinsic parameter, are solved.

[0168] Among them, the reprojection error is used to measure the calibration accuracy. It is used to characterize the pixel distance between the 3D point projected onto the 2D point on the image through the intrinsic and extrinsic parameters and the actual detected 2D corner point. For example, the average reprojection error is less than 0.5 pixels and the maximum error is less than 2 pixels.

[0169] In one possible embodiment, determining the third extrinsic parameter of the other camera based on the second extrinsic parameter and the first extrinsic parameter of the standard camera includes: determining the fourth extrinsic parameter of the other camera based on the second extrinsic parameter and the first extrinsic parameter of the standard camera; and determining the third extrinsic parameter based on the first extrinsic parameter and the fourth extrinsic parameter of the other camera.

[0170] Specifically, the rotation matrix of the fourth extrinsic parameter is determined based on the rotation matrices of the first and second extrinsic parameters. That is, the absolute attitude of the other cameras relative to the world is determined based on the attitudes of the other cameras relative to the standard camera and the attitude of the standard camera relative to the world.

[0171] Specifically, the rotation matrix refers to the attitude transformation of the coordinate system. Based on the matrix multiplication rule, the attitude of other cameras relative to the world is the product of the attitude of other cameras relative to the standard camera and the attitude of the standard camera relative to the world. That is, the rotation matrix of the fourth extrinsic parameter is the product of the rotation matrix of the first extrinsic parameter and the rotation matrix of the second extrinsic parameter.

[0172] Specifically, the translation vector of the fourth extrinsic parameter is determined based on the translation vectors of the first and second extrinsic parameters. That is, the absolute position of the other camera origins relative to the world is determined based on the relative positions of the other camera origins with respect to the standard camera origin, and the absolute position of the standard camera origin relative to the world.

[0173] Specifically, the translation vector refers to the position transformation of the origin of the coordinate system. The absolute position of the origin of other cameras relative to the world is equal to the sum of the relative position of the origin of other cameras relative to the standard camera origin and the absolute position of the standard camera origin relative to the world. That is, the translation vector of the fourth extrinsic parameter is the sum of the translation vector of the first extrinsic parameter and the translation vector of the second extrinsic parameter.

[0174] Specifically, by leveraging coordinate transformation transitivity, the first extrinsic parameter of the standard camera coordinate system relative to the world coordinate system is cascaded with the second extrinsic parameter of other camera coordinate systems relative to the standard camera coordinate system to derive the fourth extrinsic parameter of other cameras. This reflects the relative attitude / position relationship between other cameras and the standard camera, avoiding relative attitude confusion when other cameras calculate extrinsic parameters independently, and providing a stable parameter basis for subsequent 3D measurement and target localization.

[0175] The first extrinsic parameter of each of the other cameras is directly calculated and does not depend on the parameters of other cameras. It is used to reflect the absolute attitude trend of the camera relative to the world coordinate system. The fourth extrinsic parameter of each of the other cameras is indirectly derived and is used to ensure the consistency of relative attitude among the multiple cameras.

[0176] For example, based on the first and fourth extrinsic parameters of other cameras, the average deviation between the theoretical pixel coordinates and the actual detection coordinates is calculated, and the reliability of the first and fourth extrinsic parameters is evaluated based on the average deviation. The higher the reliability, the greater the weight.

[0177] Specifically, based on the corresponding weights, the rotation matrix and translation vector in the first and fourth extrinsic parameters are weighted and fused to obtain the third extrinsic parameters of other cameras.

[0178] Among them, the intrinsic and extrinsic parameters of multi-view cameras include the first intrinsic and first extrinsic parameters of the standard camera, and the first intrinsic and third extrinsic parameters of other cameras.

[0179] As can be seen, in the embodiments of this application, by fusing the extrinsic parameters of other cameras, the error of a single extrinsic parameter can be offset, so that the final extrinsic parameter reflects both the absolute world benchmark and ensures the relative consistency of multiple cameras, thereby improving the accuracy of the extrinsic parameters of other cameras.

[0180] As can be seen, in this embodiment, grayscale information in the cross-sectional direction perpendicular to the extension direction of the grid line is collected at different locations of the grid line. The center line of each grid line is determined by collecting multiple grayscale information, and the coordinates of the intersection of any two center lines are calculated. The corner points of the checkerboard are replaced by the coordinates of the intersection of the center lines, thereby solving the defect of corner point detection failure caused by defocusing in the microscopic scene. At the same time, even if the calibration plate image does not include all the intersection points, this application can determine the position of all intersection points based on multiple center lines, thereby completing the calibration of the multi-view camera and improving the accuracy of camera calibration in the microscopic scene.

[0181] In one possible embodiment, please refer to Figure 5 , Figure 5 This is a flowchart illustrating another method for calibrating a multi-view camera on a stereo microscope, as provided in an embodiment of this application. Figure 5 As shown, first, the calibration plate is placed within the microscope's field of view. Then, the microscope's binocular camera can be used to record images from different poses. Afterward, denoising processing is performed on the grid calibration plate images recorded by the binocular camera to remove particle noise, thereby reducing errors caused by subsequent grid line fitting.

[0182] Furthermore, a Fourier transform operation is performed on the obtained calibration board image to obtain the spatial spectrum of the image, and the orientation of the grid lines is obtained by analyzing the obtained spatial spectrum in order to solve for the center line of the grid lines in the subsequent process.

[0183] Furthermore, based on the orientation of the grid lines, a cross-section of the grid lines is taken, and the cross-section is fitted using the Butworth function to obtain a fitting curve that characterizes the gray-scale distribution of the grid line cross-section.

[0184] Specifically, the advantage of the Butworth function lies in its ripple-free amplitude-frequency response curve, which can suppress local noise interference while preserving the gray-scale gradient trend.

[0185] Furthermore, the midpoint of the fitted curve is determined, and the gradient of the grayscale cross-section after median filtering is calculated to find the location with the largest grayscale change rate. This location is the sub-pixel center point of the grid line cross-section. To ensure positioning accuracy, it is usually necessary to extract the center points of multiple cross-sections of the same grid line, forming a discrete set of sub-pixel coordinate center points.

[0186] In one possible embodiment, the midpoint of the fitted curve can be determined by solving for the two endpoints of the full width and half height of the fitted curve, and then calculating the median of the coordinates of the two endpoints as the midpoint of the grid line in that section.

[0187] Specifically, the discrete sub-pixel center points of the same grid line should theoretically be distributed along a straight line. However, due to factors such as imaging noise and optical distortion, the actual coordinate points may have slight deviations. Therefore, a mathematical model of the grid line centerline needs to be established using curve fitting methods to accurately determine the spatial direction of the line.

[0188] Furthermore, the center line of the grid line is obtained by least squares fitting. Specifically, let the coordinates of the discrete sub-pixel center points of a certain grid line be {(x1, y1), (x2, y2), ..., (x... n y n Assuming the equation of the centerline is y=kx+b, by minimizing the sum of the squares of the perpendicular distances from each point to the line, we can solve for the optimal parameters k and b, and obtain the mathematical expression of the grid centerline.

[0189] Furthermore, calculate the intersection points of the grid lines, take the mathematical expression of the center lines of any two grid lines, and calculate the intersection points.

[0190] Repeat this step to obtain the coordinates of all grid line intersections.

[0191] Furthermore, based on the coordinates of all grid line intersections, the intrinsic and extrinsic parameters of the stereo camera can be obtained.

[0192] As can be seen, in this embodiment, the Butworth function is used to solve for the center point of the grid line cross section on the calibration plate, and the mathematical expression of the center line of the grid line is obtained by fitting the center points of multiple cross sections of each grid line. Then, the sub-pixel coordinates of the intersection points of the center lines of each grid line are calculated. Finally, by comparing the distribution law of the sub-pixel coordinates of the above intersection points with the theoretical spatial distribution law of these intersection points, the solution of the camera's intrinsic and extrinsic parameters is completed.

[0193] This application's embodiments employ a method of detecting the intersection points of grid centerlines, replacing the checkerboard corner detection commonly used in traditional camera calibration. In actual camera calibration scenarios, if the calibration board image is out of focus, it becomes difficult to accurately locate corner points, and detection may even fail. However, in this application's embodiments, the grid centerline intersection points are extracted using a Butworth function, and the centerlines are fitted before solving the problem. Even if the corner points are out of focus, the coordinates of all intersection points can be stably obtained, effectively avoiding the calibration inaccuracies caused by out-of-focus areas.

[0194] Meanwhile, traditional methods require capturing complete checkerboard images to obtain all corner points. If the calibration board image fails to include all corner points due to limitations in shooting angle and field of view, it can lead to decreased accuracy or failure in parameter solving. However, in the embodiments of this application, as long as the image contains most of the grid lines, the centerline equation can be fitted to extend the calculation and derive the coordinates of the intersection points of the grid line centerlines in the uncaptured areas. This ensures the completeness and accuracy of the camera's intrinsic and extrinsic parameters, significantly improving the flexibility and robustness of the calibration process.

[0195] For examples consistent with the above embodiments, please refer to... Figure 6 , Figure 6 This is a functional unit block diagram of a multi-view camera calibration device for a stereo microscope provided in an embodiment of this application, as shown below. Figure 6 As shown, the multi-view camera calibration device 60 for the stereo microscope includes:

[0196] The first acquisition unit 61 is used to acquire images of the calibration board when the calibration board is in different poses, wherein the calibration board is constructed by multiple regularly arranged grid lines;

[0197] The first determining unit 62 is used to determine the extension direction of each grid line in the calibration plate image;

[0198] The second acquisition unit 63 is used to acquire multiple grayscale information of multiple target areas at multiple preset positions of the grid line based on a cross-sectional direction perpendicular to the extension direction.

[0199] The second determining unit 64 is used to determine the center line of the corresponding grid line based on the plurality of grayscale information;

[0200] The third determining unit 65 is used to determine the coordinates of the intersection point between any two of the center lines;

[0201] The fourth determining unit 66 is used to determine the intrinsic and extrinsic parameters of the multi-view camera based on the coordinates of the intersection point.

[0202] In one possible embodiment, the second determining unit 64 is specifically used for determining the center line of the corresponding grid line based on the plurality of grayscale information:

[0203] Multiple grayscale information values ​​are fitted to obtain multiple target fitting curves;

[0204] Determine multiple fitting midpoints of multiple target fitted curves;

[0205] The centerline is obtained by fitting multiple midpoints.

[0206] In one possible embodiment, in fitting multiple grayscale information to obtain multiple target fitting curves, the second determining unit 64 is further configured to:

[0207] Determine the target angle formed by the grid lines and the edge of the calibration board image;

[0208] Obtain the width of the grid lines;

[0209] A first function is constructed based on the target angle, the width, and the initial control parameters;

[0210] The grayscale information is fitted according to the first function to obtain an initial fitting curve;

[0211] Determine the fitting effect of the initial fitted curve;

[0212] The initial control parameters are optimized based on the fitting effect to obtain the target control parameters;

[0213] The first function is modified according to the target control parameters to obtain the second function;

[0214] The second function is used to fit multiple grayscale information values ​​to obtain multiple target fitting curves.

[0215] In one possible embodiment, the second determining unit 64 is further configured to: construct the first function based on the target angle, the width, and the initial control parameters;

[0216] Based on the target angle and the initial control parameters, a third function is constructed, which is used to characterize the distance between a pixel in a single target region and the center line of the corresponding grid line;

[0217] Determine the ratio of the third function to the width;

[0218] The first function is constructed based on the ratio.

[0219] In one possible embodiment, the fourth determining unit 66 is further configured to: determine the intrinsic and extrinsic parameters of the multi-view camera based on the intersection coordinates.

[0220] Based on the intersection coordinates and the preset calibration algorithm, determine the first intrinsic parameter and the first extrinsic parameter of each camera in the multi-view camera;

[0221] Acquire the standard camera from the multi-view cameras;

[0222] Based on the first intrinsic parameters of the standard camera and the first intrinsic parameters of the other cameras, the second extrinsic parameters of the other cameras relative to the standard camera are determined, wherein the other cameras are cameras other than the standard camera among the multi-view cameras;

[0223] Based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, determine the third extrinsic parameter of the other cameras;

[0224] The intrinsic and extrinsic parameters of the multi-camera are obtained based on the first intrinsic parameter of the standard camera, the first extrinsic parameter of the standard camera, the first intrinsic parameter of the other cameras, and the third extrinsic parameter of the other cameras.

[0225] In one possible embodiment, in determining the third extrinsic parameter of the other camera based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, the fourth determining unit 66 is further configured to:

[0226] Based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, determine the fourth extrinsic parameter of the other cameras;

[0227] The third extrinsic parameter is determined based on the first extrinsic parameter and the fourth extrinsic parameter of the other cameras.

[0228] In one possible embodiment, in determining multiple midpoints of the multiple target fitted curves, the first determining unit 62 is further configured to:

[0229] Determine multiple height medians of the multiple target fitted curves;

[0230] Among the multiple target fitting curves, multiple width pairs corresponding to the multiple median heights are determined;

[0231] The plurality of fitted midpoints are determined based on the plurality of width pairs.

[0232] It is understood that since the method embodiments and the device embodiments are different presentations of the same technical concept, the content of the method embodiment section in this application should be adapted to the device embodiment section in a synchronous manner, and will not be repeated here.

[0233] In the case of using integrated units, please refer to Figure 7 , Figure 7 This is a functional unit block diagram of another stereo microscope multi-view camera calibration device provided in the embodiments of this application, such as... Figure 7As shown, the multi-view camera calibration device 60 for a stereo microscope includes a processing module 602 and a communication module 601. The processing module 602 controls and manages the operation of the multi-view camera calibration device 60, for example, executing the steps of the first acquisition unit 61, the first determination unit 62, the second acquisition unit 63, the second determination unit 64, the third determination unit 65, and the fourth determination unit 66, and / or performing other processes of the technology described herein. The communication module 601 is used for interaction between the multi-view camera calibration device 60 and other devices.

[0234] Among them, such as Figure 7 As shown, the multi-view camera calibration device 60 of the stereo microscope may also include a storage module 603, which is used to store the program code and data of the multi-view camera calibration device 60 of the stereo microscope.

[0235] The processing module 602 can be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0236] The communication module 601 can be a transceiver, RF circuit, or communication interface, etc. The storage module 603 can be a memory.

[0237] All relevant content in each scenario involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here. The multi-view camera calibration device 60 of the stereo microscope described above can perform the above... Figure 2 The method for calibrating the multi-view camera of the stereo microscope is shown.

[0238] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device proposed in an embodiment of this application, such as... Figure 8As shown, the electronic device 800 includes a processor 810, a memory 820, a communication interface 830, and one or more programs 821. The one or more programs 821 are stored in the memory and configured to be executed by the processor. When the program is executed, it includes some or all of the steps of the multi-view camera calibration method of any stereomicroscope described in the above method embodiments. The processor, memory, and communication interface are interconnected and complete communication between them.

[0239] The memory can be volatile memory such as Dynamic Random Access Memory (DRAM) or non-volatile memory such as a hard disk drive. The memory stores a set of executable program code, and the processor calls the executable program code stored in the memory to execute some or all of the steps of any stereo microscope multi-view camera calibration method described in the above embodiments of the stereo microscope multi-view camera calibration method.

[0240] As can be seen, the electronic device 800 described in this application embodiment first acquires calibration board images when the calibration board is in different poses, the calibration board being constructed by multiple regularly arranged grid lines; determines the extension direction of each grid line in the calibration board image; at multiple preset positions of the grid lines, based on the cross-sectional direction perpendicular to the extension direction, acquires multiple grayscale information of multiple target areas; determines the center line of the corresponding grid line according to the multiple grayscale information; determines the coordinates of the intersection point between any two of the center lines; and determines the intrinsic and extrinsic parameters of the multi-view camera according to the intersection point coordinates.

[0241] This application collects grayscale information in the cross-sectional direction perpendicular to the extension direction of the grid lines at different locations along the grid lines. By collecting multiple grayscale information, the center line of each grid line is determined, and the coordinates of the intersection of any two center lines are calculated. The coordinates of the intersection of the center lines are used to replace the corner points of the checkerboard, thereby solving the defect of corner point detection failure caused by defocusing in microscopic scenes. At the same time, even if the calibration board image does not include all intersection points, this application can determine the positions of all intersection points based on multiple center lines, thereby completing the calibration of multi-view cameras and improving the accuracy of camera calibration in microscopic scenes.

[0242] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0243] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0244] It should be noted that, for the sake of simplicity, the aforementioned methods are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are optional, and the actions and modules involved are not necessarily essential to this application.

[0245] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0246] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0247] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0248] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0249] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0250] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0251] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for calibrating a multi-view camera on a stereo microscope, characterized in that, include: Acquire calibration board images when the calibration board is in different poses. The calibration board is constructed by multiple regularly arranged grid lines. The calibration board includes multiple grids, each grid is of uniform size and has a regular layout. The grid lines form a parallelogram, rectangle or square. The calibration board is characterized by black and white distinct edges and sharp corners. The extension direction of each grid line in the calibration board image is determined; wherein, the target spatial spectrum of the calibration board image is determined; the extension direction of each grid line in the calibration board image is determined based on the target spatial spectrum; wherein, a Fourier transform operation is performed on the denoised calibration board image, and the periodic gray-level changes in the spatial domain are converted into discrete frequency components in the frequency domain through the Fourier transform operation to obtain frequency peaks, which are used to characterize bright spots in the spatial spectrum; the generated target spatial spectrum is analyzed to obtain the extension direction of each grid line; wherein, the extension direction of the grid line in the spatial domain is the same as the angle of the bright spot relative to the origin in the frequency domain; At multiple preset positions of the grid line, multiple grayscale information of multiple target regions is obtained based on a cross-sectional direction perpendicular to the extension direction; wherein, multiple preset positions of the grid line are determined according to the length or width of the grid in the calibration plate; wherein, a first angle is calculated between the extension direction of the grid line and the bottom edge of the calibration plate image; and a second angle is calculated between the extension direction of the grid line and the side edge of the calibration plate image; if the first angle is less than the second angle, the preset position is determined according to the length of the grid in the calibration plate; if the first angle is greater than the second angle, the preset position is determined according to the width of the grid in the calibration plate; wherein, at each preset position of the grid line, the grayscale information of the target region in the cross-sectional direction is obtained according to the extension direction of the grid line; The center line of the corresponding grid line is determined based on the multiple grayscale information; wherein, the multiple grayscale information is fitted to obtain multiple target fitting curves; multiple fitting midpoints of the multiple target fitting curves are determined; the center line is obtained by fitting the multiple fitting midpoints; wherein, the target angle formed by the grid line and the edge of the calibration board image is determined; the width of the grid line is obtained; a first function is constructed based on the target angle, the width, and initial adjustment parameters; the grayscale information is fitted based on the first function to obtain an initial fitting curve; the fitting effect of the initial fitting curve is determined; the initial adjustment parameters are optimized based on the fitting effect to obtain target adjustment parameters; the first function is corrected based on the target adjustment parameters to obtain a second function; the multiple grayscale information is fitted based on the second function to obtain multiple target fitting curves; Determine the coordinates of the intersection point between any two of the centerlines; The intrinsic and extrinsic parameters of the multi-view camera are determined based on the coordinates of the intersection point.

2. The method according to claim 1, characterized in that, The construction of the first function based on the target angle, the width, and the initial adjustment parameters includes: Based on the target angle and the initial control parameters, a third function is constructed, which is used to characterize the distance between a pixel in a single target region and the center line of the corresponding grid line; Determine the ratio of the third function to the width; The first function is constructed based on the ratio.

3. The method according to claim 1, characterized in that, The step of determining the intrinsic and extrinsic parameters of the multi-view camera based on the intersection point coordinates includes: Based on the intersection coordinates and the preset calibration algorithm, determine the first intrinsic parameter and the first extrinsic parameter of each camera in the multi-view camera; Acquire the standard camera from the multi-view cameras; Based on the first intrinsic parameters of the standard camera and the first intrinsic parameters of other cameras, determine the second extrinsic parameters of other cameras relative to the standard camera, wherein the other cameras are cameras other than the standard camera among the multi-view cameras; Based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, determine the third extrinsic parameter of the other cameras; The intrinsic and extrinsic parameters of the multi-camera are obtained based on the first intrinsic parameter of the standard camera, the first extrinsic parameter of the standard camera, the first intrinsic parameter of the other cameras, and the third extrinsic parameter of the other cameras.

4. The method according to claim 3, characterized in that, The step of determining the third extrinsic parameter of the other cameras based on the second extrinsic parameter and the first extrinsic parameter of the standard camera includes: Based on the second extrinsic parameter and the first extrinsic parameter of the standard camera, determine the fourth extrinsic parameter of the other cameras; The third extrinsic parameter is determined based on the first extrinsic parameter and the fourth extrinsic parameter of the other cameras.

5. The method according to claim 1, characterized in that, Determining multiple midpoints of the multiple target fitted curves includes: Determine multiple height medians of the multiple target fitted curves; Among the multiple target fitting curves, multiple width pairs corresponding to the multiple median heights are determined; The plurality of fitted midpoints are determined based on the plurality of width pairs.

6. A multi-view camera calibration device for a stereo microscope, characterized in that, include: The first acquisition unit is used to acquire images of the calibration board when it is in different poses. The calibration board is constructed by multiple regularly arranged grid lines. The calibration board includes multiple grids, each grid is of uniform size and has a regular layout. The grid formed by the grid lines is a parallelogram, rectangle or square. The calibration board is characterized by black and white distinct edges and sharp corners. The first determining unit is used to determine the extension direction of each grid line in the calibration board image; wherein, the target spatial spectrum of the calibration board image is determined; the extension direction of each grid line in the calibration board image is determined based on the target spatial spectrum; wherein, a Fourier transform operation is performed on the denoised calibration board image, and the periodic gray-level changes in the spatial domain are converted into discrete frequency components in the frequency domain through the Fourier transform operation to obtain frequency peaks, which are used to characterize bright spots in the spatial spectrum; the generated target spatial spectrum is analyzed to obtain the extension direction of each grid line; wherein, the extension direction of the grid line in the spatial domain is the same as the angle of the bright spot relative to the origin in the frequency domain; The second acquisition unit is configured to acquire multiple grayscale information of multiple target regions at multiple preset positions of the grid line, based on a cross-sectional direction perpendicular to the extension direction; wherein, the multiple preset positions of the grid line are determined according to the length or width of the grid in the calibration plate; wherein, a first angle is calculated between the extension direction of the grid line and the bottom edge of the calibration plate image; and a second angle is calculated between the extension direction of the grid line and the side edge of the calibration plate image; if the first angle is less than the second angle, the preset position is determined according to the length of the grid in the calibration plate; if the first angle is greater than the second angle, the preset position is determined according to the width of the grid in the calibration plate; wherein, at each preset position of the grid line, the grayscale information of the target region in the cross-sectional direction is acquired according to the extension direction of the grid line; The second determining unit is configured to determine the center line of a corresponding grid line based on the plurality of grayscale information; wherein, the plurality of grayscale information is fitted to obtain a plurality of target fitting curves; a plurality of fitting midpoints of the plurality of target fitting curves are determined; the center line is obtained by fitting the plurality of fitting midpoints; wherein, the target angle formed by the grid line and the edge of the calibration plate image is determined; the width of the grid line is obtained; a first function is constructed based on the target angle, the width, and initial adjustment parameters; the grayscale information is fitted based on the first function to obtain an initial fitting curve; the fitting effect of the initial fitting curve is determined; the initial adjustment parameters are optimized based on the fitting effect to obtain target adjustment parameters; the first function is corrected based on the target adjustment parameters to obtain a second function; and the plurality of grayscale information is fitted based on the second function to obtain the plurality of target fitting curves. The third determining unit is used to determine the coordinates of the intersection point between any two of the center lines; The fourth determining unit is used to determine the intrinsic and extrinsic parameters of the multi-view camera based on the coordinates of the intersection point.

7. An electronic device, characterized in that, The device includes: The system includes a memory, a processor, and executable program code stored in the memory and executable on the processor, wherein the processor executes the executable program code to perform the steps of the multi-view camera calibration method for a stereo microscope as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable program code, which includes execution instructions for performing the steps of the multi-view camera calibration method for a stereo microscope as described in any one of claims 1-5.

Images