Multi-camera motion capture method based on fuzzy control and dynamic synchronization

Abstract

The invention relates to the technical field of camera control, in particular to a multi-view camera motion capturing method based on fuzzy control and dynamic synchronization. The method comprises the following steps: calculating a farthest identification distance, and placing a multi-view camera within the farthest identification distance; the multi-view camera captures a moving image and uploads the image to the fuzzy control system, and the fuzzy control system calculates to obtain an adjustment value according to the tracking deviation of the target object and adjusts the position and speed of the tracking moving target, so that the tracking moving target tracks the target object; a tracking error is calculated and transmitted to a fuzzy control system, and the fuzzy control system adjusts the camera to track the target object according to the tracking error and the error change rate EC of the moving target; the method has the advantages that the reliability of time synchronization is ensured by adopting a high-quality time signal source through a signal quality evaluation algorithm, and the time alignment of the cameras is ensured and the capture precision is improved by adopting a dynamic synchronization algorithm of timestamps.

Description

Technical Field

[0001] The present invention relates to the field of camera control technology, and in particular to a multi-camera motion capture method based on fuzzy control and dynamic synchronization; the method is based on fuzzy control technology and dynamic synchronization algorithm, and is mainly used in fields such as video surveillance, motion analysis, virtual reality, and augmented reality that require high-precision, multi-view capture of moving objects. Background Art

[0002] Due to the focusing problem of cameras in a multi-camera environment, the traditional focusing method has a slow response speed, inaccurate focus on moving objects, and cannot capture effective motion images in a timely and effective manner; in addition, there is a clock error in the time alignment of multiple cameras, and the accuracy and stability of multiple cameras cannot be controlled. There is an urgent need for a method that can use multiple cameras to effectively capture moving objects and improve control accuracy and stability.

[0003] For example, a Chinese patent with application number 202020835422.1 and application date of May 18, 2020, is a utility model patent named "A target tracking and controlling camera system". Its technical solution is: a target tracking and controlling camera system, characterized in that: it includes a camera module for shooting, and a servo pan-tilt for adjusting the angle of the camera module; the camera module is arranged on the servo pan-tilt; the camera module includes a camera lens, an image processing module for identifying and processing images, and a fuzzy controller for controlling; the camera lens tracks the target; the camera lens, the image processing module, the fuzzy controller and the servo pan-tilt are communicatively connected in sequence; although the above patent can realize the function of tracking camera.

[0004] However, the above patent cannot achieve real-time synchronization of multiple multi-eye cameras, and cannot provide stable pulse signals through real-time signal quality evaluation to ensure the reliability of time synchronization. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a multi-camera motion capture method based on fuzzy control and dynamic synchronization for multiple multi-cameras to synchronously capture moving people or objects with high precision.

[0006] In order to achieve the above technical effects, the technical solution of this application is as follows:

[0007] A multi-camera motion capture method based on fuzzy control and dynamic synchronization includes the following steps:

[0008] Step 1: Calculate the farthest recognition distance of multiple multi-view cameras, and place the multiple multi-view cameras within the farthest recognition distance of the moving object or person to be captured;

[0009] Step 2: When the object or person starts to move, the multi-eye camera captures the moving image of the moving object or person, and uploads the image to the fuzzy control system. The fuzzy control system calculates the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd according to the position deviation e of the target object and the speed deviation Δe of the target object. The controller installed in the multi-eye camera adjusts the position and speed of the moving target according to the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd calculated by the fuzzy control system, so that it tracks the target object;

[0010] Step 3: According to the position and speed of the moving target, the tracking error of the moving target is calculated, and then the error is transmitted to the fuzzy control system, which uses the tracking error E of the moving target to calculate the tracking error E of the moving target. t And the error change rate EC, adjust the camera's pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd, thereby adjusting the position and speed of the tracking moving target again to make it track the target object.

[0011] Furthermore, the maximum recognition distance D for capturing a moving object or person is calculated. The specific formula is as follows:

[0012]

[0013] Where D max is the maximum recognition distance, f is the focal length of the lens, H is the object height, and p is the pixel size.

[0014] Further, calculating the farthest recognition distance D of capturing a moving person or object includes the following steps:

[0015] Step a: Obtain the minimum recognition distance d1;

[0016] Step b: Obtain the maximum target surface width L2;

[0017] Step c: obtaining a single pixel resolution δ;

[0018] Step d: Obtain the farthest recognition distance D according to the obtained minimum recognition distance d1, the maximum target width L2 and the single pixel resolution δ;

[0019] In step a, the specific formula for obtaining the minimum recognition distance d1 is:

[0020]

[0021] Where L1 is the width of the camera target surface, and θ is the lens divergence angle;

[0022] In step b, the specific formula for obtaining the maximum target width L2 is:

[0023]

[0024] Where, L1 is the width of the camera target surface, d1 is the minimum recognition distance, L2 is the maximum target surface width, and d2 is the incremental distance;

[0025] In step c, the specific formula for calculating the single pixel resolution δ is:

[0026]

[0027] In the formula, R is the horizontal resolution of the camera, and L2 is the maximum target width;

[0028] The specific formula for calculating the farthest recognition distance D in step d is:

[0029]

[0030] Where L1 is the width of the camera target surface, θ is the lens divergence angle, P is the minimum recognition pixel, and L 人 is the horizontal size of the face, and R is the horizontal resolution of the camera.

[0031] Furthermore, the specific steps of calculating the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd in step 2 are as follows:

[0032] Step a: fuzzify the position deviation e of the target object and the speed deviation Δe of the target object, convert the precise input value into a fuzzy value, perform fuzzy reasoning based on fuzzy rules, and calculate the fuzzy output value;

[0033] Step b: defuzzifying the calculated fuzzy output value and converting it into an accurate pitch angle adjustment value Δθ, a horizontal rotation angle adjustment value Δφ and an up and down movement distance adjustment value Δd;

[0034] In the step a, the fuzzification process is to obtain the membership values ​​of different fuzzy sets according to the membership function for the position deviation e of the target object and the speed deviation Δe of the target object, wherein the fuzzy sets include: negative large (NB), negative medium (NM), negative small (NS), zero (ZE), positive small (PS), positive medium (PM), and positive large (PB);

[0035] In step 2, the fuzzy control system converts the calculated pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd into a pulse signal, and the pulse signal is used to transmit a signal to the camera to adjust the direction and focal length of the camera; after receiving the pulse signal, the controller in the camera performs corresponding adjustments on the camera, each pulse signal corresponds to a step, and these step angles are accumulated to adjust the viewing angle and focal length of the camera; the controller is a stepper motor, and after the camera receives the pulse signal, the stepper motor controls the camera to rotate to a corresponding angle according to the signal frequency, signal quantity and signal strength, and changes the direction or focal length of the camera; the signal strength is used to control the rotation force of the stepper motor to prevent over-rotation or under-rotation; the pulse signal contains control information and a synchronization mark, and the control information content includes timestamp information, which is used to record the generation time of the pulse signal to ensure the synchronization between multiple cameras; the synchronization mark is used to ensure that each camera is adjusted at the same time point to ensure the consistency of the captured action; the pulse signal is sent using a pulse high-stability synchronization clock source installed on a Beidou chip or GPS chip in each camera to provide a unified synchronization clock signal for multiple multi-eye cameras.

[0036] Furthermore, the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd are used to adjust and optimize the tracking of the camera, and then the position that the optimized camera needs to move is compared with the actual position to obtain the tracking error E of the fuzzy control system. t and the error change rate EC, and adjust the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd through the fuzzy control system. During the movement process, adjust the camera tracking again according to the actual position to be moved;

[0037] Tracking Error E t Using the mobile target tracking algorithm, let the actual position of the target at time t be P t , the predicted position of the target at time t is The tracking error E t The specific formula is as follows:

[0038]

[0039] Where P t is the actual position vector of the target at time t, usually expressed as a three-dimensional coordinate (x t ,y t , z t ); is the predicted position vector of the target at time t, usually expressed as a three-dimensional coordinate E t is the tracking error vector at time t, expressed as (e x , ey , e z ),in: Where e x is the tracking error of the target on the x-axis, e y is the tracking error of the target on the y-axis, e z is the tracking error of the target in the z-axis, x t is the actual position of the target on the x-axis at time t, y t is the actual position of the target on the y axis at time t, z t is the actual position of the target on the z-axis at time t, is the predicted position of the target on the x-axis at time t, is the predicted position of the target on the y-axis at time t, is the predicted position of the target on the z-axis at time t;

[0040] From the images captured by multiple multi-cameras, the actual position P of the target is obtained through image processing algorithms t =(x t ,y t , z t ), based on the target’s previous motion trajectory and speed information, the dynamic model is used to predict the target’s position at time t The specific formula for prediction is as follows:

[0041]

[0042] Where P t-1 is the actual position of the target at time t-1, V t-1 is the velocity vector of the target at time t-1, expressed as (v x , v y , v z ), Δt is the time interval t-(t-1);

[0043] The error change rate EC is the tracking error E t The rate of change over time, the specific formula is as follows:

[0044]

[0045] In the formula, EC represents the speed of error change. According to the tracking error E t The size of the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd are adjusted according to the size of and the direction of change of the error change rate EC.

[0046] Furthermore, the pulse high-stable synchronous clock source receives the pulse sources sent by Beidou and GPS respectively, calculates the mean and variance of the count values ​​of the pulse sources sent by Beidou and GPS in the same time, compares the variance of the pulse sources sent by Beidou and GPS, evaluates the quality of Beidou and GPS signals through a signal quality evaluation algorithm, adjusts the output time information of the chip, finds the optimal pulse source to assist in switching the signal source, and outputs a pulse signal; the pulse high-stable synchronous clock source sends the pulse signal to multiple multi-eye cameras wirelessly, and a signal conversion module is provided inside the multiple multi-eye cameras to convert the pulse signal into a control instruction of a stepper motor, and the control instruction controls the rotation direction, speed and rotation angle of the stepper motor; the pulse signal determines the time to generate the pulse signal through a signal quality evaluation algorithm, and when the signal quality Q exceeds the set threshold, a pulse signal is generated.

[0047] Furthermore, after the pulse signal is generated, the local timestamp and remote timestamp received within the synchronization period are first collected, and the time offset ΔT is calculated through the timestamp synchronization algorithm. Finally, the original timestamp Treceived received for each frame is adjusted to the adjusted timestamp Tadjusted for each frame, ensuring time alignment so that the frames captured by the camera are aligned in time, and multiple multi-eye cameras can synchronously track and capture moving objects in real time.

[0048] The local timestamps and remote timestamps received during the synchronization period are collected, and the time offset ΔT is calculated through the timestamp synchronization algorithm to align the frames captured by each camera in time. The specific formula is as follows:

[0049] T adjusted =T received +ΔT

[0050] Where, T adjusted is the adjusted timestamp to ensure time alignment; T received is the original timestamp received; ΔT is the time offset;

[0051] The specific solution formula for the time offset is as follows:

[0052]

[0053] Where, T local，i is the local timestamp received for the ith time; T received，i is the remote timestamp received for the ith time, and N is the noise intensity.

[0054] Furthermore, the signal quality evaluation algorithm is to evaluate the quality of Beidou and GPS signals and provide a pulse signal according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows:

[0055] Q = α × SNR + β × SS - γ × MP

[0056] In the formula, Q is the signal quality evaluation value, which is used to judge the quality of the signal. The larger the value, the better the signal quality. α is the weight coefficient of the signal-to-noise ratio, which usually ranges from 0 to 1 and is used to adjust the influence of the signal-to-noise ratio on the overall signal quality. β is the weight coefficient of the signal strength, which usually ranges from 0 to 1 and is used to adjust the influence of the signal strength on the overall signal quality. γ is the weight coefficient of the multipath interference, which usually ranges from 0 to 1 and is used to adjust the negative impact of the multipath interference on the overall signal quality. SS is the signal strength, which indicates the power strength of the received signal. MP is the multipath interference coefficient, which is used to quantify the degree of signal interference caused by the multipath effect. SNR is the signal-to-noise ratio, which indicates the ratio of signal strength to noise, usually expressed in decibels. The higher the signal-to-noise ratio, the better the signal quality.

[0057] Furthermore, the specific formula for obtaining the signal-to-noise ratio SNR is:

[0058]

[0059] Where P signal is the signal power, P noise is the noise power;

[0060] Measuring the time it takes for a signal to travel from a satellite to a receiver is used to determine the length of the signal transmission path. The specific formula for signal arrival time is:

[0061] TOA = t received -t transmitted

[0062] Where TOA is the signal arrival time, t received is the time of receiving the signal, t transmitted The time when the signal is sent;

[0063] The clock deviation is obtained to determine the difference between the receiver's internal clock and the satellite clock. The specific formula is:

[0064] B=t receiver -t satellite

[0065] Where B is the clock deviation, t receiver is the receiver time, t satellite is the satellite time;

[0066] The multipath effect metric is used to determine the distortion or delay of the signal caused by reaching the receiver through multiple paths. The specific formula is as follows:

[0067]

[0068] Where M is the multipath effect measure, A i is the amplitude of the ith path, φ i is the amplitude of the ith path, N is the number of paths;

[0069] Calculate the signal delay, which is used to determine the time delay from the satellite to the receiver. The specific formula is:

[0070]

[0071] Where L is the signal delay, d is the signal transmission distance, c is the speed of light, and Δt is other delay factors;

[0072] Based on the above-obtained signal-to-noise ratio, signal arrival time, signal delay, clock deviation and multipath effect measurement results, it is determined whether to select Beidou or GPS as the signal source, so as to output the optimal pulse source.

[0073] Furthermore, the signal quality evaluation algorithm is to evaluate the quality of Beidou and GPS signals and provide a pulse signal according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows:

[0074]

[0075] In the formula, Q is the signal quality, which reflects the reliability of the signal; SS is the signal strength, which indicates the received signal power strength; N is the noise strength, which indicates the received background noise power.

[0076] According to the above technical solution, the present invention has the following beneficial effects:

[0077] 1. The present invention adopts a high-quality time signal source through a signal quality evaluation algorithm, provides a stable pulse signal, ensures the reliability of time synchronization, and adopts a dynamic synchronization algorithm of the timestamp to dynamically adjust the timestamp by calculating the time offset, thereby ensuring the time alignment of each camera and improving the capture accuracy.

[0078] 2. The present invention simultaneously deploys Beidou and GPS chips in multi-cameras, enhances the signal receiving capability and anti-interference capability of the system, improves the reliability and accuracy of time synchronization, can effectively solve the time synchronization problem in the multi-camera system, and ensures the time alignment of the captured frames of multiple cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0081] Example 1

[0082] A multi-camera motion capture method based on fuzzy control and dynamic synchronization includes the following steps:

[0083] Step 1: Calculate the farthest recognition distance of multiple multi-view cameras, and place the multiple multi-view cameras within the farthest recognition distance of the moving object or person to be captured;

[0084] Step 2: When the object or person starts to move, the multi-eye camera captures the moving image of the moving object or person, and uploads the image to the fuzzy control system. The fuzzy control system calculates the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd according to the position deviation e of the target object and the speed deviation Δe of the target object. The controller installed in the multi-eye camera adjusts the position and speed of the moving target according to the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd calculated by the fuzzy control system, so that it tracks the target object;

[0085] Step 3: According to the position and speed of the moving target, the tracking error of the moving target is calculated, and then the error is transmitted to the fuzzy control system, which uses the tracking error E of the moving target to calculate the tracking error E of the moving target. t And the error change rate EC, adjust the camera's pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd, thereby adjusting the position and speed of the tracking moving target again to make it track the target object.

[0086] Calculate the farthest recognition distance D for capturing moving objects or people. The specific formula is as follows:

[0087]

[0088] Where D max is the maximum recognition distance, f is the focal length of the lens, H is the object height, and p is the pixel size.

[0089] Example 2

[0090] A multi-camera motion capture method based on fuzzy control and dynamic synchronization includes the following steps:

[0091] Step 1: Calculate the farthest recognition distance of multiple multi-view cameras, and place the multiple multi-view cameras within the farthest recognition distance of the moving object or person to be captured;

[0092] Step 2: When the object or person starts to move, the multi-eye camera captures the moving image of the moving object or person, and uploads the image to the fuzzy control system. The fuzzy control system calculates the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd according to the position deviation e of the target object and the speed deviation Δe of the target object. The controller installed in the multi-eye camera adjusts the position and speed of the moving target according to the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd calculated by the fuzzy control system, so that it tracks the target object;

[0093] Step 3: According to the position and speed of the moving target, the tracking error of the moving target is calculated, and then the error is transmitted to the fuzzy control system, which uses the tracking error E of the moving target to calculate the tracking error E of the moving target. t And the error change rate EC, adjust the camera's pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd, thereby adjusting the position and speed of the tracking moving target again to make it track the target object.

[0094] Calculating the farthest recognition distance D of a person or object in motion includes the following steps:

[0095] Step a: Obtain the minimum recognition distance d1;

[0096] Step b: Obtain the maximum target surface width L2;

[0097] Step c: obtaining a single pixel resolution δ;

[0098] Step d: Obtain the farthest recognition distance D according to the obtained minimum recognition distance d1, the maximum target width L2 and the single pixel resolution δ;

[0099] In step a, the specific formula for obtaining the minimum recognition distance d1 is:

[0100]

[0101] Where L1 is the width of the camera target surface, and θ is the lens divergence angle;

[0102] In step b, the specific formula for obtaining the maximum target width L2 is:

[0103]

[0104] Where, L1 is the width of the camera target surface, d1 is the minimum recognition distance, L2 is the maximum target surface width, and d2 is the incremental distance;

[0105] In step c, the specific formula for calculating the single pixel resolution δ is:

[0106]

[0107] In the formula, R is the horizontal resolution of the camera, and L2 is the maximum target width;

[0108] The specific formula for calculating the farthest recognition distance D in step d is:

[0109]

[0110] Where L1 is the width of the camera target surface, θ is the lens divergence angle, P is the minimum recognition pixel, and L 人 is the horizontal size of the face, and R is the horizontal resolution of the camera.

[0111] Example 3

[0112] Based on Example 1 or Example 2, the specific steps of calculating the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd in step 2 are as follows:

[0113] Step a: fuzzify the position deviation e of the target object and the speed deviation Δe of the target object, convert the precise input value into a fuzzy value, perform fuzzy reasoning based on fuzzy rules, and calculate the fuzzy output value;

[0114] Step b: Defuzzify the calculated fuzzy output value and convert it into an accurate pitch angle adjustment value Δθ, a horizontal rotation angle adjustment value Δφ and an up and down movement distance adjustment value Δd. The specific solution formula for the pitch angle adjustment value Δθ is as follows:

[0115]

[0116] The specific solution formula for the horizontal rotation angle adjustment value Δφ is as follows:

[0117]

[0118] The specific solution formula for the up and down movement distance adjustment value Δd is as follows:

[0119]

[0120] In the above formula, is the membership degree of the fuzzy reasoning result, zi is the specific value of the fuzzy output variable, i is the number of the reasoning rule, and the specific examples of the fuzzy rules are shown in Table 2;

[0121] Table 2 shows the rule numbers and fuzzy rule definitions.

[0122]

[0123]

[0124] In step a, the fuzzification process is to obtain the membership values ​​of different fuzzy sets according to the membership function for the position deviation e of the target object and the speed deviation Δe of the target object, wherein the fuzzy sets include: negative large (NB), negative medium (NM), negative small (NS), zero (ZE), positive small (PS), positive medium (PM), and positive large (PB);

[0125] As shown in step a, the position deviation e of the target object and the speed deviation Δe of the target object are first inputted and fuzzy processed, as shown in Table 1.

[0126] Table 1 is the definition of fuzzy sets and membership functions.

[0127]

[0128]

[0129] In step 2, the fuzzy control system solves the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down moving distance adjustment value Δd, and the specific steps are as follows: six fuzzy variables are used, and the six fuzzy variables include adjusting the upper pitch angle, the lower pitch angle, the left horizontal rotation angle, the right horizontal rotation angle, the upper moving distance, and the lower moving distance; each fuzzy variable is divided into multiple fuzzy subsets;

[0130] Fuzzy subsets of fuzzy variables (m=64n=6 in the following formula):

[0131]

[0132] As parameter set

[0133]

[0134] Parameter sets can be simplified Among them, R1 represents the first fuzzy inference rule, R m represents the mth fuzzy inference rule; x n Represents the input variable of the system, which is the value of the actual system state; u 1 ,u 2 , ..., u m is the output result of the fuzzy inference rule. Each u i According to the corresponding rule R m The calculated output; These are constant term parameters in each inference rule and are not related to the input variables; is each input variable x1, x2, ..., x n The corresponding coefficients, p1, p2, ..., p n because The coefficients p1, p2, ..., p n The simplified parameter set represents the unified weight of each input variable in all rules; the coefficients of the same input variable in different fuzzy rules are unified into one parameter set, which simplifies the parameter structure of the system, reduces the complexity of the model, and makes the adjustment of parameters and the optimization of the system simpler and more intuitive; the membership function of the fuzzy control set maps the input variables to the fuzzy set, quantifies the uncertainty, and provides a basis for fuzzy reasoning and decision-making; two fuzzy subsets are used to control the target; each fuzzy variable uses three parameters to determine the membership function of the "positive" or "negative" fuzzy control set, which is used to adjust and optimize the camera tracking. The specific formula is as follows:

[0135]

[0136] In the formula, u Ai (χ) represents the membership of the input variable χ to the fuzzy set Ai, which ranges from 0 to 1 and represents the degree of membership of the variable χ in the fuzzy set Ai; χ is the actual value of the input variable; c p is the center position of the fuzzy set "positive", which determines the input value corresponding to the highest membership degree of the membership function; c n is the center position of the fuzzy set “negative”; a p The expansion parameter in the fuzzy set "positive" determines the "width" or "expansion" of the membership function and controls the range of the fuzzy set; a n is the extended parameter in the fuzzy set “negative”, used for the “negative” fuzzy set; b p is the shape parameter of the fuzzy set “positive”, which controls the shape and change rate of the membership function; b n is the shape parameter of the “negative” fuzzy set, used for “negative” fuzzy sets.

[0137] The fuzzy control system adjusts the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down moving distance adjustment value Δd, and the input variables to the fuzzy set are {NB, NM, NS, Z, PS, PM, PB}. The output variables are the fuzzy control rules of the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down moving distance adjustment value Δd. The fuzzy set of the input variables {NB, NM, NS, Z, PS, PM, PB} is converted into corresponding values. The 49 values ​​of the fuzzy control rules of the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down moving distance adjustment value Δd are stored respectively, and the table is looked up in the form of base address and shift. Then, the fuzzy reasoning is used for defuzzification to obtain the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down moving distance adjustment value Δd; among which NB is negative and large, NM is negative and medium, NS is negative and small, Z is zero, PS is positive and small, PM is positive and medium, and PB is positive and large in the fuzzy set.

[0138] The above contents related to the fuzzy control system are the prior art in this field, and the above solution methods are only part of the solution methods in the prior art.

[0139] In step 2, the fuzzy control system converts the calculated pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd into a pulse signal. The pulse signal is used to transmit a signal to the multi-eye camera to adjust the direction and focal length of the camera. After receiving the pulse signal, the controller in the camera performs corresponding adjustments on the camera. Each pulse signal corresponds to a step, and these step angles are accumulated to adjust the viewing angle and focal length of the camera. The controller is a stepper motor. After the camera receives the pulse signal, the stepper motor controls the camera to rotate to a corresponding angle and changes the direction or focal length of the camera according to the signal frequency, signal quantity and signal strength. The signal strength is used to control the stepper motor. The rotation force of the machine to prevent over-rotation or under-rotation; the pulse signal contains control information and synchronization identification. The control information content includes timestamp information, which is used to record the generation time of the pulse signal to ensure the synchronization between multiple cameras; the synchronization identification is used to ensure that each camera is adjusted at the same time point to ensure the consistency of captured actions; the synchronization identification is special information embedded in the pulse signal, and its main functions are to unify the time, coordinate the actions of multiple cameras, prevent signal interference and conflict, and accurately align the time; the pulse signal is sent using a pulse high-stability synchronization clock source installed on the Beidou chip or GPS chip in each camera to provide a unified synchronization clock signal for multiple multi-eye cameras.

[0140] Example 4

[0141] Based on Example 3, the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd are used to adjust and optimize the tracking of the camera, and then the position that the optimized camera needs to move is compared with the actual position to obtain the tracking error E of the fuzzy control system. t and the error change rate EC, and adjust the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd through the fuzzy control system. During the movement process, adjust the camera tracking again according to the actual position to be moved;

[0142] Tracking Error E t Using the mobile target tracking algorithm, let the actual position of the target at time t be P t , the predicted position of the target at time t is The tracking error E t The specific formula is as follows:

[0143]

[0144] Where P t is the actual position vector of the target at time t, usually expressed as a three-dimensional coordinate (x t ,y t , z t ); is the predicted position vector of the target at time t, usually expressed as a three-dimensional coordinate E t is the tracking error vector at time t, expressed as (e x , e y , e z ),in: Where e x is the tracking error of the target on the x-axis, e y is the tracking error of the target on the y-axis, e z is the tracking error of the target in the z-axis, x t is the actual position of the target on the x-axis at time t, y t is the actual position of the target on the y axis at time t, z t is the actual position of the target on the z-axis at time t, is the predicted position of the target on the x-axis at time t, is the predicted position of the target on the y-axis at time t, is the predicted position of the target on the z-axis at time t;

[0145] From the images captured by multiple multi-cameras, the actual position P of the target is obtained through image processing algorithms t =(x t ,y t ,zt ), based on the target’s previous motion trajectory and speed information, the dynamic model is used to predict the target’s position at time t The specific formula for prediction is as follows:

[0146]

[0147] Where P t-1 is the actual position of the target at time t-1, V t-1 is the velocity vector of the target at time t-1, expressed as (v x , v y , v z ), Δt is the time interval t-(t-1);

[0148] The error change rate EC is the tracking error E t The rate of change over time, the specific formula is as follows:

[0149]

[0150] In the formula, EC represents the speed of error change. According to the tracking error E t The size of the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd are adjusted according to the size of and the direction of change of the error change rate EC.

[0151] Example 5

[0152] Based on Example 4, a pulse high-stable synchronous clock source receives pulse sources sent by Beidou and GPS respectively, and the pulse source transmission can be through a local area network, such as WIFI / LORA signals, calculates the mean and variance of the count values ​​of the pulse sources sent by Beidou and GPS in the same time, compares the variance of the pulse sources sent by Beidou and GPS, evaluates the quality of Beidou and GPS signals through a signal quality evaluation algorithm, adjusts the output time information of the chip, finds the optimal pulse source to assist in switching the signal source, and outputs a pulse signal; the pulse high-stable synchronous clock source sends the pulse signal to multiple multi-eye cameras wirelessly, and a signal conversion module is provided inside the multiple multi-eye cameras to convert the pulse signal into a control instruction of a stepper motor, and the control instruction controls the rotation direction, speed and rotation angle of the stepper motor; the pulse signal determines the time to generate the pulse signal through a signal quality evaluation algorithm, and when the signal quality Q exceeds the set threshold, a pulse signal is generated.

[0153] After the pulse signal is generated, the local timestamp and remote timestamp received within the synchronization period are first collected. The time offset ΔT is calculated through the timestamp synchronization algorithm. Finally, the original timestamp Treceived received for each frame is adjusted to the adjusted timestamp Tadjusted for each frame. This ensures that the frames captured by the camera are aligned in time, and multiple multi-eye cameras can synchronize and track moving objects in real time.

[0154] The local timestamps and remote timestamps received during the synchronization period are collected, and the time offset ΔT is calculated through the timestamp synchronization algorithm to align the frames captured by each camera in time. The specific formula is as follows:

[0155] T adjusied =T received +ΔT

[0156] Where, T adjusted is the adjusted timestamp to ensure time alignment; T received is the original timestamp received; ΔT is the time offset;

[0157] The specific solution formula for the time offset is as follows:

[0158]

[0159] Where, T local，i is the local timestamp received for the ith time; T received，i is the remote timestamp received for the ith time, and N is the noise intensity.

[0160] The signal quality evaluation algorithm evaluates the quality of Beidou and GPS signals and provides pulse signals according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows:

[0161] Q = α × SNR + β × SS - γ × MP

[0162] In the formula, Q is the signal quality evaluation value, which is used to judge the quality of the signal. The larger the value, the better the signal quality. α is the weight coefficient of the signal-to-noise ratio, which usually ranges from 0 to 1 and is used to adjust the influence of the signal-to-noise ratio on the overall signal quality. β is the weight coefficient of the signal strength, which usually ranges from 0 to 1 and is used to adjust the influence of the signal strength on the overall signal quality. γ is the weight coefficient of the multipath interference, which usually ranges from 0 to 1 and is used to adjust the negative impact of the multipath interference on the overall signal quality. SS is the signal strength, which indicates the power strength of the received signal. MP is the multipath interference coefficient, which is used to quantify the degree of signal interference caused by the multipath effect. SNR is the signal-to-noise ratio, which indicates the ratio of signal strength to noise, usually expressed in decibels. The higher the signal-to-noise ratio, the better the signal quality. The signal-to-noise ratio and signal strength are important indicators of signal quality, measuring the clarity of the signal relative to the background noise, while the signal strength directly affects the receivability of the signal.

[0163] The specific formula for obtaining the signal-to-noise ratio SNR is:

[0164]

[0165] Where P signal is the signal power, P noise is the noise power;

[0166] Measuring the time it takes for a signal to travel from a satellite to a receiver is used to determine the length of the signal transmission path. The specific formula for signal arrival time is:

[0167] TOA = t received -t transmitted

[0168] Where TOA is the signal arrival time, t received is the time of receiving the signal, t transmitted The time when the signal is sent;

[0169] The clock deviation is obtained to determine the difference between the receiver's internal clock and the satellite clock. The specific formula is:

[0170] B=t receiver -t satellite

[0171] Where B is the clock deviation, t receiver is the receiver time, t satellite is the satellite time;

[0172] The multipath effect metric is used to determine the distortion or delay of the signal caused by reaching the receiver through multiple paths. The specific formula is as follows:

[0173]

[0174] Where M is the multipath effect measure, A i is the amplitude of the ith path, φ i is the amplitude of the ith path, N is the number of paths;

[0175] Calculate the signal delay, which is used to determine the time delay from the satellite to the receiver. The specific formula is:

[0176]

[0177] Where L is the signal delay, d is the signal transmission distance, c is the speed of light, and Δt is other delay factors;

[0178] Based on the above-obtained signal-to-noise ratio, signal arrival time, signal delay, clock deviation and multipath effect measurement results, it is determined whether to select Beidou or GPS as the signal source, so as to output the optimal pulse source.

[0179] The signal quality evaluation algorithm evaluates the quality of Beidou and GPS signals and provides pulse signals according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows:

[0180]

[0181] In the formula, Q is the signal quality, which reflects the reliability of the signal; SS is the signal strength, which indicates the received signal power strength; N is the noise strength, which indicates the received background noise power.

[0182] The signal quality assessment algorithm is used to ensure the use of high-quality time signal sources to ensure the reliability of time synchronization; a dynamic synchronization algorithm for timestamps is used to dynamically adjust timestamps by calculating time offsets to ensure time alignment of each camera and improve capture accuracy; Beidou and GPS chips are deployed simultaneously in a multi-camera system to enhance the system's signal reception and anti-interference capabilities, improve the reliability and accuracy of time synchronization, and effectively solve the time synchronization problem in a multi-camera system to ensure time alignment of frames captured by multiple cameras.

[0183] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A multi-camera motion capture method based on fuzzy control and dynamic synchronization, characterized in that: The steps include: Step 1: Calculate the farthest recognition distance of multiple multi-view cameras, and place the multiple multi-view cameras within the farthest recognition distance of the moving object or person to be captured; Step 2: When the object or person starts to move, the multi-eye camera captures the moving image of the moving object or person, and uploads the image to the fuzzy control system. The fuzzy control system calculates the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd according to the position deviation e of the target object and the speed deviation Δe of the target object. The controller installed in the multi-eye camera adjusts the position and speed of the moving target according to the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd calculated by the fuzzy control system, so that it tracks the target object; Step 3: According to the position and speed of the moving target, the tracking error of the moving target is calculated, and then the error is transmitted to the fuzzy control system, which uses the tracking error E of the moving target to calculate the tracking error E of the moving target. t And the error change rate EC, adjust the camera's pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd, thereby adjusting the position and speed of the tracking moving target again to make it track the target object.

2. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 1, characterized in that: Calculate the farthest recognition distance D for capturing moving objects or people. The specific formula is as follows: Where D max is the maximum recognition distance, f is the focal length of the lens, H is the object height, and p is the pixel size.

3. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 1, characterized in that: Calculating the farthest recognition distance D of a person or object in motion includes the following steps: Step a: Obtain the minimum recognition distance d1; Step b: Obtain the maximum target surface width L2; Step c: obtaining a single pixel resolution δ; Step d: Obtain the farthest recognition distance D according to the obtained minimum recognition distance d1, the maximum target width L2 and the single pixel resolution δ; In step a, the specific formula for obtaining the minimum recognition distance d1 is: Where L1 is the width of the camera target surface, and θ is the lens divergence angle; In step b, the specific formula for obtaining the maximum target width L2 is: Where, L1 is the width of the camera target surface, d1 is the minimum recognition distance, L2 is the maximum target surface width, and d2 is the incremental distance; In step c, the specific formula for calculating the single pixel resolution δ is: In the formula, R is the horizontal resolution of the camera, and L2 is the maximum target width; The specific formula for calculating the farthest recognition distance D in step d is: Where L1 is the width of the camera target surface, θ is the lens divergence angle, P is the minimum recognition pixel, and L 人 is the horizontal size of the face, and R is the horizontal resolution of the camera.

4. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 1, characterized in that: The specific steps for calculating the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ, and the up and down movement distance adjustment value Δd in step 2 are as follows: Step a: fuzzify the position deviation e of the target object and the speed deviation Δe of the target object, convert the precise input value into a fuzzy value, perform fuzzy reasoning based on fuzzy rules, and calculate the fuzzy output value; Step b: defuzzifying the calculated fuzzy output value and converting it into an accurate pitch angle adjustment value Δθ, a horizontal rotation angle adjustment value Δφ and an up and down movement distance adjustment value Δd; In the step a, the fuzzification process is to obtain membership values ​​of different fuzzy sets according to the membership function for the position deviation e of the target object and the speed deviation Δe of the target object, wherein the fuzzy sets include: negative large, negative medium, negative small, zero, positive small, positive medium and positive large; In step 2, the fuzzy control system converts the calculated pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd into a pulse signal, and the pulse signal is used to transmit a signal to the camera to adjust the direction and focal length of the camera; after receiving the pulse signal, the controller in the camera performs corresponding adjustments on the camera, each pulse signal corresponds to a step, and these step angles are accumulated to adjust the viewing angle and focal length of the camera; the controller is a stepper motor, and after the camera receives the pulse signal, the stepper motor controls the camera to rotate to a corresponding angle according to the signal frequency, signal quantity and signal strength, and changes the direction or focal length of the camera; the signal strength is used to control the rotation force of the stepper motor to prevent over-rotation or under-rotation; the pulse signal contains control information and a synchronization mark, and the control information content includes timestamp information, which is used to record the generation time of the pulse signal to ensure the synchronization between multiple cameras; the synchronization mark is used to ensure that each camera is adjusted at the same time point to ensure the consistency of the captured action; the pulse signal is sent using a pulse high-stability synchronization clock source installed on a Beidou chip or GPS chip in each camera to provide a unified synchronization clock signal for multiple multi-eye cameras.

5. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 1, characterized in that: The pitch angle adjustment value Δθ, horizontal rotation angle adjustment value Δφ and up and down movement distance adjustment value Δd are used to adjust and optimize the camera tracking. Then the position that the optimized camera needs to move is compared with the actual position to obtain the tracking error E of the fuzzy control system. t and the error change rate EC, and adjust the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd through the fuzzy control system. During the movement process, adjust the camera tracking again according to the actual position to be moved; Tracking Error E t Using the mobile target tracking algorithm, let the actual position of the target at time t be P t , the predicted position of the target at time t is The tracking error E t The specific formula is as follows: Where P t is the actual position vector of the target at time t, usually expressed as a three-dimensional coordinate (x t ,y t , z t ); is the predicted position vector of the target at time t, usually expressed as a three-dimensional coordinate E t is the tracking error vector at time t, expressed as (e x , e y , e z ),in: Where e x is the tracking error of the target on the x-axis, e y is the tracking error of the target on the y-axis, e z is the tracking error of the target in the z-axis, x t is the actual position of the target on the x-axis at time t, y t is the actual position of the target on the y axis at time t, z t is the actual position of the target on the z-axis at time t, is the predicted position of the target on the x-axis at time t, is the predicted position of the target on the y-axis at time t, is the predicted position of the target on the z-axis at time t; From the images captured by multiple multi-cameras, the actual position P of the target is obtained through image processing algorithms t =(x t ,y t , z t ), based on the target’s previous motion trajectory and speed information, the dynamic model is used to predict the target’s position at time t The specific formula for prediction is as follows: Where P t-1 is the actual position of the target at time t-1, V t-1 is the velocity vector of the target at time t-1, expressed as (v x , v y , v z ), Δt is the time interval t-(t-1); The error change rate EC is the tracking error E t The rate of change over time, the specific formula is as follows: In the formula, EC represents the speed of error change. According to the tracking error E t The size of the pitch angle adjustment value Δθ, the horizontal rotation angle adjustment value Δφ and the up and down movement distance adjustment value Δd are adjusted according to the size of and the direction of change of the error change rate EC.

6. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 4, characterized in that: The pulse high-stability synchronous clock source receives the pulse sources sent by Beidou and GPS respectively, calculates the mean and variance of the count values ​​of the pulse sources sent by Beidou and GPS in the same time, compares the variance of the pulse sources sent by Beidou and GPS, evaluates the quality of Beidou and GPS signals through the signal quality evaluation algorithm, adjusts the output time information of the chip, finds the optimal pulse source to assist in switching the signal source, and outputs the pulse signal; The pulse high-stability synchronous clock source sends the pulse signal to multiple multi-eye cameras wirelessly. The multiple multi-eye cameras are equipped with a signal conversion module to convert the pulse signal into the control instruction of the stepper motor. The control instruction controls the rotation direction, speed and rotation angle of the stepper motor. The pulse signal determines the time to generate the pulse signal through the signal quality evaluation algorithm. When the signal quality Q exceeds the set threshold, a pulse signal is generated.

7. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 6, characterized in that: After the pulse signal is generated, the local timestamp and remote timestamp received within the synchronization period are first collected. The time offset ΔT is calculated through the timestamp synchronization algorithm. Finally, the original timestamp Treceived received for each frame is adjusted to the adjusted timestamp Tadjusted for each frame. This ensures that the frames captured by the camera are aligned in time, and multiple multi-eye cameras can synchronize and track moving objects in real time. The local timestamps and remote timestamps received during the synchronization period are collected, and the time offset ΔT is calculated through the timestamp synchronization algorithm to align the frames captured by each camera in time. The specific formula is as follows: T adjusted =T received +ΔT Where, T adjusted For the adjusted timestamps, ensure time alignment; T received is the original timestamp received; ΔT is the time offset; The specific solution formula for the time offset is as follows: Where, T local，i is the local timestamp received for the i-th time; T received，i is the remote timestamp received for the ith time, and N is the noise intensity.

8. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 6, characterized in that: The signal quality evaluation algorithm is used to evaluate the quality of Beidou and GPS signals, and provides a pulse signal according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows: Q = α × SNR + β × SS - γ × MP In the formula, Q is the signal quality evaluation value, which is used to judge the quality of the signal. The larger the value, the better the signal quality. α is the weight coefficient of the signal-to-noise ratio, which usually ranges from 0 to 1 and is used to adjust the influence of the signal-to-noise ratio on the overall signal quality. β is the weight coefficient of signal strength, and its value range is usually between 0 and 1. It is used to adjust the influence of signal strength on the overall signal quality; γ is the weight coefficient of multipath interference, and its value range is usually between 0 and 1. It is used to adjust the negative impact of multipath interference on the overall signal quality; SS is signal strength, which indicates the power strength of the received signal; MP is the multipath interference coefficient, which is used to quantify the degree of signal interference caused by the multipath effect; SNR is the signal-to-noise ratio, which indicates the ratio of signal strength to noise, and is usually expressed in decibels. The higher the signal-to-noise ratio, the better the signal quality.

9. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 8, characterized in that: The specific formula for obtaining the signal-to-noise ratio SNR is: Where P signal is the signal power, P noise is the noise power; Measuring the time it takes for a signal to travel from a satellite to a receiver is used to determine the length of the signal transmission path. The specific formula for signal arrival time is: TOA=t received +t transmited Where TOA is the signal arrival time, t received is the time of receiving the signal, t transmitted The time when the signal is sent; The clock deviation is obtained to determine the difference between the receiver's internal clock and the satellite clock. The specific formula is: B=t receiver -t satellite Where B is the clock deviation, t receiver is the receiver time, t satellite is the satellite time; The multipath effect metric is used to determine the distortion or delay of the signal caused by reaching the receiver through multiple paths. The specific formula is as follows: Where M is the multipath effect measure, A i is the amplitude of the ith path, φ i is the amplitude of the ith path, N is the number of paths; Calculate the signal delay, which is used to determine the time delay from the satellite to the receiver. The specific formula is: Where L is the signal delay, d is the signal transmission distance, c is the speed of light, and Δt is other delay factors; Based on the above-obtained signal-to-noise ratio, signal arrival time, signal delay, clock deviation and multipath effect measurement results, it is determined whether to select Beidou or GPS as the signal source, so as to output the optimal pulse source.

10. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 6, characterized in that: The signal quality evaluation algorithm is used to evaluate the quality of Beidou and GPS signals, and provides a pulse signal according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows: In the formula, Q is the signal quality, which reflects the reliability of the signal; SS is the signal strength, which indicates the received signal power strength; N is the noise strength, which indicates the received background noise power.

Images