A multi-view vision system master camera selection method in optical motion capture

Abstract

The present application belongs to the field of computer vision, and proposes a kind of main camera selection method of multi-view vision system in optical motion capture, and the device involved includes Marker point for identifying target, industrial camera, POE network switch, PC and two-dimensional calibration board or one-dimensional calibration rod for calibrating camera parameters.The present application fully considers error distribution factors of camera network in the calibration process of motion capture system, especially for the main camera selection problem of multi-view vision system composed of multiple cameras, uses weighted undirected graph to obtain the shortest connected path between camera nodes, uses the weighted average of back projection error and mean square deviation to construct evaluation function, to evaluate the rationality of reference camera selection.The present application not only makes the selection of main camera in the calibration process of motion capture system more scientific and reasonable, and error control more effective, but also the whole process is automatically completed in PC, without manual operation, and has higher efficiency and ease of use.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method for selecting the main camera in a multi-view vision system for optical motion capture. Background Technology

[0002] A motion measurement system based on optical motion capture can "freeze" the motion of a target within the camera's field of view onto an image. Then, using 3D reconstruction methods, the spatial coordinates of the target object can be calculated, and the motion can be decomposed into six degrees of freedom to accurately obtain the target object's motion information. This technology has played a significant role in marine engineering model experiments due to its advantages of being interference-free, highly accurate, and suitable for large-scale scenarios.

[0003] Optical motion capture systems are based on a network of multiple cameras (also known as a multi-view vision system). The parameters of each camera are obtained by calibrating this vision system. Then, the spatial coordinates of the measured object are reconstructed using the camera parameters and the image coordinates of the captured target, and a six-degree-of-freedom motion decomposition is performed to obtain motion information. The accuracy of the calibration directly affects the data quality of the motion measurement; therefore, optimizing this process is crucial.

[0004] In the calibration process of a multi-view vision system, it is necessary not only to obtain the intrinsic parameters of each camera, but also to accurately calculate the spatial positional relationship (extrinsic parameters) between the cameras. Therefore, it is necessary to select one camera as the master camera and use its camera coordinate system as the global coordinate system of the camera network, while the coordinates of other cameras will be unified into this global coordinate system (Li Tongbin, Dong Shuai. Research on camera distribution in multi-camera global 3D measurement [J]. Optoelectronic Technology Application, 2019, 34(2):11-16.). Currently, when using motion capture systems, after completing the camera numbering, the master camera is often the camera manually numbered "1" when arranging the cameras (Zhang Guiyang, Huo Ju, Yang Ming, et al. High-precision 3D deformation full-field measurement with joint constraint optimization of multi-camera network [J]. Optics and Precision Engineering, 2021, 29(07):1653-1666.). This selection method is relatively convenient, but it lacks the support of multi-view geometry theory, so there is a lot of room for improvement in the overall error control of the system. In practice, the calibration errors of each camera node in a network composed of multi-view vision systems are different. Therefore, when performing a global unified analysis of external parameters, the reconstruction error of the entire vision system after calibration will vary greatly due to the different positions of the master camera. Thus, how to select the optimal master camera is crucial to the measurement accuracy of the optical motion capture system. Designing a master camera selection method for multi-view vision systems in optical motion capture has great application value. Summary of the Invention

[0005] To address the system calibration and optimization problem of multi-view vision systems in optical motion capture applications, this invention focuses on the design of a method for selecting the primary camera. It treats a network of multiple cameras as a weighted undirected graph for error propagation analysis and proposes a method for selecting the primary camera in multi-view vision systems for optical motion capture. This method uses a weighted undirected graph to obtain the shortest connected paths between camera nodes and constructs an evaluation function using the weighted average of backprojection error and root mean square error to assess the rationality of the reference camera selection. The proposed method provides an effective technical solution for selecting the primary camera in multi-view vision systems, significantly improving the overall calibration accuracy of motion capture systems.

[0006] The technical solution of the present invention:

[0007] A method for selecting the main camera in a multi-view vision system for optical motion capture includes the following steps:

[0008] Step A: According to the requirements of the motion measurement experiment, install marker points on the target to be measured, and complete the arrangement of M cameras (high-speed industrial cameras with network interfaces are generally used in motion capture systems). At the same time, number the cameras (the numbering only needs to meet the requirement of non-repetition), such as: 1, 2, 3...M; then, connect the cameras to the PoE switch, and the switch is also connected to the network port of the user's PC via a network cable.

[0009] Step B: Based on the principle of multi-view geometry, use a one-dimensional calibration rod or a two-dimensional calibration plate to calculate the intrinsic and extrinsic parameters of each pair of cameras, treat each camera as a node, and construct a weighted undirected graph based on the back projection error;

[0010] Step C: Select a camera as the main camera and make that camera node the master node in the undirected graph;

[0011] Step D: Find the shortest path from other camera nodes to the main camera node in the undirected graph;

[0012] Step E: Calculate the mean of the weights on each path. and the mean square error of each path weight

[0013] Step F: Based on the two set weight values ​​ω1 and ω2, calculate the Q of the camera as the main camera according to equation (1). j value;

[0014]

[0015] Step G: Repeat steps C-F until Q for each camera as the main camera is calculated. j value;

[0016] Step H: Compare the Q values ​​of each camera j Value, select Q j The camera corresponding to the smallest value is selected as the main camera. According to Q... j The selection of the primary camera not only controls the overall error but also considers the balance of error distribution among various cameras, providing better initial values ​​for the adjustment process.

[0017] A method for selecting the master camera in a multi-view vision system for optical motion capture includes marker points for target identification, several industrial cameras, a PoE network switch, a PC, and a calibration board or calibration rod for calibrating camera parameters. Each camera and the PC are connected to the network switch via network cables. The camera arrangement needs to be rationally arranged according to the motion range of the target being measured, ensuring that the target is always within the field of view of the camera network. After the equipment is installed and the motion capture system is calibrated, the images captured by the cameras participating in motion capture are first captured on the PC via the PoE network switch, and the cameras are paired. Then, based on the principle of multi-view geometry, the intrinsic and extrinsic parameters of the cameras are calculated using a two-dimensional calibration board or a one-dimensional calibration rod. Simultaneously, a weighted undirected graph of the camera network is constructed based on the back-projection error. Then, cameras are successively selected as master cameras on the undirected graph, and the evaluation value Q is calculated. Finally, the camera with the smallest Q value is selected as the master camera.

[0018] The beneficial effects of this invention are as follows: The motion capture system calibration process fully considers the error distribution factors of the camera network, especially for the main camera selection problem in a multi-view vision system composed of multiple cameras (more than four). It optimizes the error propagation path by constructing a weighted undirected graph and uses the path mean and variance to construct the main camera selection evaluation function. This invention not only makes the selection of the main camera in the motion capture system calibration process more scientific and reasonable, and the error control more effective, but also automatically completes the entire process on a PC without manual operation, resulting in higher efficiency and ease of use. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the optical motion capture system.

[0020] Figure 2 This is a schematic diagram showing the arrangement of industrial cameras.

[0021] Figure 3 These are sample data used for testing, where (a) is the sample distribution and (b) is the co-visibility relationship.

[0022] Figure 4 It is a weighted undirected graph of each node of the camera.

[0023] Figure 5It is the curve showing the change in back projection error.

[0024] In the diagram: 1. Marker point; 2. Industrial camera; 3. PoE network switch; 4. PC; 5. 2D calibration board; 6. 1D calibration rod. Detailed Implementation

[0025] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0026] A method for selecting the main camera in a multi-view vision system for optical motion capture, the structure of which is shown in the attached diagram. Figure 1 As shown:

[0027] A method for selecting the main camera in a multi-view vision system for optical motion capture comprises marker points 1 for target identification, several industrial cameras 2, a PoE network switch 3, a PC 4, and a calibration board 5 or calibration rod for calibrating camera parameters. Each industrial camera 2 and the PC 4 are connected to the network switch 3 via a network cable. The arrangement of the industrial cameras 2 needs to be rationally arranged according to the motion range of the target being measured, so that the target object is always within the field of view of the industrial camera 2 network. After the equipment is installed and the motion capture system is calibrated, firstly, the images captured by the industrial cameras 2 participating in motion capture are captured on the PC 4 through the PoE network switch 3, and the industrial cameras 2 are paired. Then, based on the principle of multi-view geometry, the intrinsic and extrinsic parameters of the industrial cameras 2 are calculated using a two-dimensional calibration board 5 or a one-dimensional calibration rod 6, and a weighted undirected graph of the industrial camera 2 network is constructed based on the back projection error. Then, industrial cameras 2 are successively selected as the main camera on the undirected graph to calculate the evaluation value Q. Finally, the industrial camera 2 with the smallest Q value is selected as the main camera. The specific method is described as follows:

[0028] Step A: According to the requirements of the motion measurement experiment, install Marker 1 on the target to be measured, and complete the arrangement of M industrial cameras 2 (high-speed industrial cameras with network interfaces are generally used in motion capture systems). At the same time, number the cameras (the numbering only needs to meet the requirement of non-repetition), such as: 1, 2, 3...M; then, connect the cameras to the PoE switch 3, and the switch 3 is also connected to the network port of the user's PC 4 via a network cable.

[0029] Step B: Based on the principle of multi-view geometry, use a one-dimensional calibration rod 6 or a two-dimensional calibration plate 5 to calculate the parameters of each pair of cameras, take each industrial camera 2 as a node and construct a weighted undirected graph based on the back projection error;

[0030] Step C: Select an industrial camera 2 as the main camera, and designate the camera node as the master node in the undirected graph;

[0031] Step D: Find the shortest path from other industrial camera nodes 2 to the main camera node in the undirected graph;

[0032] Step E: Calculate the mean of the weights on each path. and the mean square error of each path weight

[0033] Step F: Based on the two set weight values ​​ω1 and ω2, ω1 = 0.4, ω2 = 0.6; calculate the Q of the industrial camera 2 as the main camera according to equation (1). j value;

[0034]

[0035] Step G: Repeat steps C-F until the Q value of each industrial camera 2 as the main camera is calculated. j value;

[0036] Step H: Compare the Q values ​​of each industrial camera 2 j Value, select Q j The industrial camera 2 corresponding to the smallest value is selected as the main camera. According to Q... j The selection of the primary camera not only controls the overall error but also considers the balance of error distribution among various cameras, providing better initial values ​​for the adjustment process.

[0037] Example

[0038] To verify the effectiveness of the reference camera selection method proposed in this invention and its adaptability to camera arrangement, 14 motion capture industrial cameras 2 were arranged in a ring (not at equal intervals) above the test site, forming a multi-view vision system. Since the industrial cameras 2 were randomly selected, their ID numbers (derived from the digital displays of each camera after power-on) were also relatively random, thus ensuring that the industrial camera 2 IDs were not correlated with their spatial positions. Figure 2 As shown. Among these 14 cameras, industrial cameras 3, 7, 8, and 9 have a pixel resolution of 1664×1088 and are equipped with 6mm F#1.8 lenses; while the other industrial cameras 2 have a pixel resolution of 2048×2048 and are equipped with 12mm F#1.8 lenses. Meanwhile, to test the impact of the selection of the reference industrial camera 2 on the system calibration error, a one-dimensional calibration rod 6 of known size was used. The one-dimensional calibration rod 6 has three collinear spherical markers ABC (also known as marker points), with a diameter of 15.9mm. The distance between points A and B is 325mm, and the distance between points B and C is 175mm.

[0039] Images of three marker points on the one-dimensional calibration rod 6 are acquired by swinging a one-dimensional calibration rod 6 (with a swing range of approximately 5m × 5m) in the area below the industrial camera array 2. In this method, 935 standard rod positions were selected for calculation and analysis. The number of data points and co-view relationships for each industrial camera 2 are as follows: Figure 3 As shown, Figure 3 In b, the X and Y axes are the numbers of industrial cameras 2, and the Z axis uses different colors to represent the number of data samples appearing in the common view area of ​​different industrial cameras 2.

[0040] The parameters of industrial camera 2 are calculated based on the data samples, and a weighted undirected graph of the nodes of industrial camera 2 is used, such as... Figure 4 As shown.

[0041] from Figure 4 As can be seen, the error distributions after parameter recovery from industrial camera 2 are different. Figure 4 In this paper, each industrial camera 2 is used as a reference camera for shortest path analysis. The results are shown in Tables 1 and 2, where Cam represents the camera number of the reference camera. This represents the average backprojection error of all camera nodes along all paths. This represents the root mean square error of the back projection error of each camera node on the path.

[0042] Table 1: Undirected Graph Path Analysis of Industrial Camera Nodes (Industrial Cameras 1-7)

[0043]

[0044] Table 2: Undirected Graph Path Analysis of Industrial Camera Nodes (Industrial Cameras 8-14)

[0045]

[0046] Calculations show that the Q value of industrial camera 2 (number 10) is the smallest among all data points. Therefore, it is inferred that when industrial camera 2 (number 10) is used as the reference camera, the error control is relatively ideal. The change in backprojection error during the iterative process is as follows: Figure 5 As shown.

Claims

1. A method for selecting the main camera in a multi-view vision system for optical motion capture, the method comprising a marker point (1) for identifying a target, several industrial cameras (2), a PoE network switch (3), a PC (4), and a two-dimensional calibration board (5) or a one-dimensional calibration rod (6) for calibrating camera parameters; the industrial cameras (2) and the PC (4) are both connected to the PoE network switch (3) via network cables; the arrangement of the industrial cameras (2) needs to be determined according to the motion range of the target being measured, so that the target is always within the field of view of the industrial camera (2) network; characterized in that, When calibrating the motion capture system, firstly, the images captured by the industrial cameras (2) participating in motion capture are captured on the PC (4) through the PoE network switch (3), and the industrial cameras (2) are paired up; then, based on the principle of multi-view geometry, the internal and external parameters of the industrial cameras (2) are calculated using a two-dimensional calibration board (5) or a one-dimensional calibration rod (6), and a weighted undirected graph of the industrial camera (2) network is constructed according to the back projection error; then, the industrial cameras (2) are successively selected as the main cameras on the weighted undirected graph for evaluation. Q The calculation is performed; finally, the value with the smallest evaluation value is selected. Q The industrial camera (2) is used as the main camera; The specific steps are as follows: Step A: According to the motion measurement requirements, install Marker point 1 on the target object and complete the process. M Arrange the industrial cameras (2) and number them; then connect the industrial cameras (2) to the POE network switch (3), and the POE network switch (3) is also connected to the network port of the PC (4) via a network cable; Step B: Based on the principle of multi-view geometry, use a one-dimensional calibration rod (6) or a two-dimensional calibration plate (5) to calculate the parameters of each pair of cameras, and use each industrial camera (2) as a node to construct a weighted undirected graph based on the back projection error; Step C: Select an industrial camera (2) as the main camera, and make the industrial camera (2) node the main node in the weighted undirected graph; Step D: Find the shortest path from other industrial camera (2) nodes to the main camera node in the weighted undirected graph; Step E: Calculate the mean of the weights on each path. and the mean square error of each path weight ; Step F: Based on the two set weight values and According to formula (1), the industrial camera (2) is used as the main camera. value; (1) Step G: Repeat steps C-F until the calculation of each industrial camera (2) as the main camera is completed. value; Step H: Compare the various industrial cameras (2) Value, selection The industrial camera (2) with the smallest value is used as the main camera.

2. The method for selecting the main camera in a multi-view vision system for optical motion capture according to claim 1, characterized in that, =0.6， =0.4。

Images