A multi-camera motion capture method based on fuzzy control and dynamic synchronization

Abstract

The present application relates to camera control technical field, specifically to a kind of multi-view camera motion capture method based on fuzzy control and dynamic synchronization;The method steps of the present application are as follows: calculate the farthest recognition distance, place multi-view camera in the farthest recognition distance;The moving image captured by multi-view camera, upload the image to fuzzy control system, calculate the adjustment value according to the deviation of target object tracking by fuzzy control system, adjust the position and speed of tracking moving target, so that it tracks target object;Calculate tracking error, transmit error to fuzzy control system, adjust camera to make it track target object according to the tracking error E 𝑡 And error rate EC of change of moving target, make camera track target object;The advantages of the present application are that high-quality time signal source is used by signal quality evaluation algorithm, the reliability of time synchronization is guaranteed, and dynamic synchronization algorithm of time stamp is used, to ensure that the time of each camera is aligned, and the capture accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera control, in particular to a multi-camera motion capture method based on fuzzy control and dynamic synchronization; the method is based on fuzzy control technology and dynamic synchronization algorithm, mainly applied to video monitoring, motion analysis, virtual reality and augmented reality, etc. fields that require high precision, multi-view capture of moving objects. BACKGROUND

[0002] Due to the focusing problem of multiple cameras in a multi-camera environment, the traditional focusing method has slow response speed, inaccurate focusing on moving objects, and cannot capture effective motion images in time; in addition, there is clock error in the time alignment of multiple cameras, which cannot achieve the precision and stability control of multiple cameras; an effective capture method of moving objects using multiple cameras is urgently needed to improve control precision and stability.

[0003] For example, a Chinese patent with application number 202020835422.1 and application date 2020.05.18, the utility model patent with patent name "target tracking control camera system", the technical scheme is: a target tracking control camera system, characterized by: a camera module for camera shooting, and a servo cloud platform for adjusting the angle of the camera module; the camera module is arranged on the servo cloud platform; the camera module includes a camera lens, an image processing module for identifying and processing images, and a controller; the camera lens tracks the fuzzy controller of the target; the camera lens, the image processing module, the fuzzy controller and the servo cloud platform are sequentially connected in communication; the above patent can realize the function of tracking and shooting.

[0004] But the above patent cannot realize real-time synchronization of multiple multi-camera, and provide stable pulse signal through real-time signal quality evaluation, ensure the reliability of time synchronization. SUMMARY

[0005] In order to solve the problems existing in the prior art, the purpose of the present application is to provide a multi-camera motion capture method based on fuzzy control and dynamic synchronization for high-precision synchronous capture of moving people or objects.

[0006] To achieve the above technical effects, the technical scheme of the present application is as follows:

[0007] A multi-camera motion capture method based on fuzzy control and dynamic synchronization, comprising the following steps:

[0008] Step 1: calculate the farthest recognition distance of multiple multi-cameras, and place multiple multi-cameras within the farthest recognition distance of the moving object or person to be captured;

[0009] Step two: when the object or person starts to move, the multi-view camera captures the moving image of the moving object or person, uploads the image to the fuzzy control system, and calculates the pitch angle adjustment value, the horizontal rotation angle adjustment value, and the up-down movement distance adjustment value according to the position deviation of the tracking target object and the speed deviation of the tracking target object. The controller installed in the multi-view camera adjusts the position and speed of the tracking moving target according to the pitch angle adjustment value, the horizontal rotation angle adjustment value, and the up-down movement distance adjustment value calculated by the fuzzy control system, so as to track the target object.

[0010] Step three: according to the position and speed of the moving target, the tracking error of the tracking moving target is calculated, and the error is transmitted to the fuzzy control system. The fuzzy control system adjusts the pitch angle adjustment value, the horizontal rotation angle adjustment value, and the up-down movement distance adjustment value of the camera according to the tracking error of the moving target and the error change rate EC, so as to adjust the position and speed of the tracking moving target again, so as to track the target object.

[0011] Further, the farthest recognition distance D of the captured moving object or person is calculated, and the specific formula is as follows:

[0012]

[0013] In the formula, D max is the maximum recognition distance, is the lens focal length, is the object height, is the pixel size.

[0014] Further, the farthest recognition distance D of the captured moving object or person is calculated, and the specific formula is as follows:

[0015] Step a: obtain the minimum recognition distance d1;

[0016] Step b: obtain the maximum target surface width L2;

[0017] Step c: obtain the single-pixel resolution ;

[0018] Step d: according to the obtained minimum recognition distance d1, maximum target surface width L2, and single-pixel resolution , the farthest recognition distance D is obtained.​​​​​​​​​​​

[0019] The specific formula for obtaining the minimum recognition distance d1 in step a is:

[0020] ;

[0021] In the formula, L1 is the camera target surface width, and θ is the lens divergence angle.

[0022] The specific formula for obtaining the maximum target surface width L2 in step b is:

[0023] ;

[0024] In the formula, L1 is the camera target surface width, d1 is the minimum recognition distance, L2 is the maximum target surface width, is the incremental distance.

[0025] The specific formula for calculating the single-pixel resolution in step c is:

[0026] ;

[0027] In the formula, R is the camera horizontal resolution, and L2 is the maximum target surface width.

[0028] The specific formula for calculating the farthest recognition distance D in step d is:

[0029]

[0030] In the formula, L1 is the camera target surface width, is the lens divergence angle, P is the minimum recognition pixel, is the face horizontal size, and R is the camera horizontal resolution.

[0031] Further, the specific steps for calculating the pitch angle adjustment value , the horizontal rotation angle adjustment value , and the up-down movement distance adjustment value in step two are as follows:

[0032] Step a: Perform fuzzification on the position deviation of the target object and the speed deviation of the target object to convert the accurate input values into fuzzy values, perform fuzzy reasoning based on fuzzy rules, and calculate fuzzy output values.

[0033] Step b: De-fuzzification of the calculated fuzzy output values to convert them into accurate pitch angle adjustment value , horizontal rotation angle adjustment value , and up-down movement distance adjustment value .

[0034] In step a, the process of blurring processing is to calculate the position deviation of the target object and the velocity deviation of the target object According to the membership function, the membership values of different fuzzy sets are obtained, wherein the fuzzy sets include: negative big (NB), negative medium (NM), negative small (NS), zero (ZE), positive small (PS), positive medium (PM), and positive big (PB);

[0035] In step two, the fuzzy control system converts the calculated pitch angle adjustment value , the horizontal rotation angle adjustment value and the up-down movement distance adjustment value into pulse signals, which are used to transmit signals to the camera to adjust the direction and focal length of the camera; the controller in the camera receives the pulse signals and executes corresponding adjustments on the camera, and each pulse signal corresponds to one step, and the cumulative steps adjust the viewing angle and focal length of the camera; the controller is a stepper motor, and after the camera receives the pulse signals, the stepper motor controls the camera to rotate by a corresponding angle according to the signal frequency, signal quantity and signal strength, and changes the orientation or focal length of the camera; the signal strength is used to control the rotation degree of the stepper motor to prevent over-rotation or insufficient rotation; the pulse signal contains control information and synchronization identifier, the control information content includes timestamp information for recording the generation time of the pulse signal to ensure the synchronization between multiple cameras; the synchronization identifier is used to ensure that each camera adjusts at the same time point to ensure the consistency of the capture action; the pulse signal is sent by the pulse high-stability synchronization clock source installed on the Beidou chip or GPS chip in each camera, and provides a unified synchronization clock signal for multiple multi-view cameras.

[0036] Further, the obtained pitch angle adjustment value , the horizontal rotation angle adjustment value and the up-down movement distance adjustment value are used to adjust and optimize the tracking of the camera, and then the position to be moved by the optimized camera is compared with the actual position to obtain the tracking error and the error change rate EC of the fuzzy control system, and the pitch angle adjustment value , the horizontal rotation angle adjustment value and the up-down movement distance adjustment value are adjusted by the fuzzy control system, and the camera tracking is adjusted again according to the obtained actual position to be moved during the movement process;

[0037] The tracking error is calculated by using a moving target tracking algorithm, and the actual position of the target at time t is set as and the predicted position of the target at time t is , the tracking error is given by the following equation:

[0038]

[0039] where P t is the actual position vector of the target at time t, usually represented as three-dimensional coordinates ; is the predicted position vector of the target at time t, usually represented as three-dimensional coordinates ; is the tracking error vector at time t, represented as where: , , , where is the tracking error of the target in the x-axis, is the tracking error of the target in the y-axis, is the tracking error of the target in the z-axis, is the actual position of the target in the x-axis at time t, is the actual position of the target in the y-axis at time t, is the actual position of the target in the z-axis at time t, is the predicted position of the target in the x-axis at time t, is the predicted position of the target in the y-axis at time t, is the predicted position of the target in the z-axis at time t.

[0040] The actual position of the target is obtained from the captured images of multiple multi-view cameras through image processing algorithms , based on the previous motion trajectory and speed information of the target, the position of the target at time t is predicted using a dynamic model , the specific formula of the prediction implementation is as follows:

[0041]

[0042] where, is the actual position of the target at time t-1, is the speed vector of the target at time t-1, represented as , is the time interval t- (t-1);

[0043] The error change rate EC is the rate of change of the tracking error over time, and the specific formula is as follows:

[0044]

[0045] In the formula, EC represents the speed of error change, and the size of the tracking error and the change direction of the error change rate EC are used to adjust the size of the pitch angle adjustment value , the horizontal rotation angle adjustment value , and the up-and-down movement distance adjustment value .

[0046] Furthermore, the pulse high-stability synchronous clock source receives the pulse sources sent by the Beidou and GPS respectively, calculates the mean and variance of the count values of the received pulse sources sent by the Beidou and GPS in the same time, compares the variances of the pulse sources sent by the Beidou and GPS, evaluates the quality of the Beidou and GPS signals through the signal quality evaluation algorithm, adjusts the output time information of the chip, finds out the optimal pulse source to assist the switching of the signal source, and outputs the pulse signal; the pulse high-stability synchronous clock source sends the pulse signal to multiple multi-view cameras through a wireless way, the multiple multi-view cameras are internally provided with a signal conversion module, the pulse signal is converted into a control instruction of a stepping motor, and the control instruction controls the rotation direction, speed and rotation angle of the stepping motor; the pulse signal is determined to be generated through the signal quality evaluation algorithm, and when the signal quality Q exceeds a set threshold value, the pulse signal is generated.

[0047] Furthermore, after the pulse signal is generated, the local time stamp and the remote time stamp received in a synchronization cycle are collected first, the time offset is calculated through a time stamp synchronization algorithm, and finally the original time stamp received by each frame is adjusted to obtain an adjusted time stamp for each frame, so as to ensure time alignment and make the frames captured by the cameras time-aligned, and complete the synchronous real-time tracking and capturing of the moving object by the multiple multi-view cameras.

[0048] The local time stamp and the remote time stamp received in a synchronization cycle are collected, the time offset is calculated through a time stamp synchronization algorithm, so that the frames captured by the cameras are time-aligned, and the specific formula is as follows:

[0049]

[0050] In the formula, is the adjusted time stamp, and time alignment is ensured; is the original time stamp received; is the time offset;

[0051] The specific solving formula of the time offset is as follows:

[0052]

[0053] In the formula, is the local time stamp received for the i-th time;​ is the i-th received remote timestamp, is the number of times the timestamp is received;

[0054] Further, the signal quality evaluation algorithm is used to evaluate the quality of Beidou and GPS signals, and provides a pulse signal according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows:

[0055]

[0056] In the formula, is the signal quality evaluation value, which is used to judge the quality of the signal, and the larger the value is, the better the signal quality is; is the weight coefficient of the signal-to-noise ratio, which is usually in the range of 0 to 1, and is used to adjust the influence degree of the signal-to-noise ratio on the overall signal quality; is the weight coefficient of the signal strength, which is usually in the range of 0 to 1, and is used to adjust the influence degree of the signal strength on the overall signal quality; is the weight coefficient of the multipath interference, which is usually in the range of 0 to 1, and is used to adjust the negative influence degree of the multipath interference on the overall signal quality; SS is the signal strength, which represents the power strength of the received signal; is the multipath interference coefficient, which is used to quantify the signal interference degree caused by the multipath effect; is the signal-to-noise ratio, which represents the ratio of the signal strength to the noise, and is usually expressed in decibels. The higher the signal-to-noise ratio is, the better the signal quality is.

[0057] Further, the signal-to-noise ratio is calculated as follows:

[0058]

[0059] In the formula, is the signal power, is the noise power;

[0060] The time of the signal arriving at the receiver from the satellite is measured to determine the length of the signal transmission path. The specific formula of the signal arrival time is as follows:

[0061]

[0062] In the formula, is the signal arrival time, is the time of receiving the signal, is the time of sending the signal;

[0063] The clock bias is calculated to judge the difference between the internal clock of the receiver and the satellite clock. The specific formula is as follows:

[0064]

[0065] wherein, is the clock bias, is the receiver time, is the satellite time;

[0066] The multipath effect measure is obtained for judging the distortion or delay of the signal caused by the multiple paths to the receiver, and the specific formula is as follows:

[0067]

[0068] wherein, is the multipath effect measure, is the amplitude of the ith path, is the amplitude of the ith path, is the number of paths;

[0069] The signal delay is obtained for judging the time delay from the satellite sending to the receiver receiving, and the specific formula is as follows:

[0070]

[0071] wherein, is the signal delay, is the signal transmission distance, is the speed of light, is other delay factors;

[0072] According to the results of the signal-to-noise ratio, the signal arrival time, the signal delay, the clock bias and the multipath effect measure obtained above, it is judged to select the signal source of Beidou or GPS, so as to output the optimal pulse source.

[0073] Further, the signal quality evaluation algorithm is used for evaluating the quality of the Beidou and GPS signals, and provides the pulse signal according to the signal quality Q, and the specific formula of the signal quality evaluation algorithm is as follows:

[0074]

[0075] wherein, Q is the signal quality, indicating the reliability of the signal; SS is the signal strength, indicating the received signal power strength; is the noise strength, indicating the received background noise power.

[0076] According to the above technical scheme, the application has the beneficial effects of:

[0077] 1. This invention uses a high-quality time signal source through a signal quality assessment algorithm to provide a stable pulse signal, ensuring the reliability of time synchronization. Furthermore, it employs a dynamic timestamp synchronization algorithm to dynamically adjust the timestamp by calculating the time offset, ensuring time alignment of each camera and improving capture accuracy.

[0078] 2. This invention deploys BeiDou and GPS chips simultaneously in a multi-camera system, enhancing the system's signal reception and anti-interference capabilities, improving the reliability and accuracy of time synchronization, effectively solving the time synchronization problem in multi-camera systems, and ensuring time alignment of frames captured by multiple cameras. Attached Figure Description

[0079] Figure 1 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0081] Example 1

[0082] A multi-camera motion capture method based on fuzzy control and dynamic synchronization includes the following steps:

[0083] Step 1: Calculate the farthest recognition distance of multiple multi-view cameras, and place multiple multi-view cameras within the farthest recognition distance of the moving object or person to be captured;

[0084] Step Two: When an object or person begins to move, the multi-view camera captures the motion image of the moving object or person and uploads the image to the fuzzy control system. The fuzzy control system then tracks the position deviation of the target object. and the velocity deviation of the target object The pitch angle adjustment value was calculated. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The controller installed inside the multi-view camera adjusts the pitch angle based on the value calculated by the fuzzy control system. Horizontal rotation angle adjustment value and vertical movement distance adjustment value Adjust the position and speed of the moving target to make it track the target object;

[0085] Step three: according to the position and speed of the moving target, calculate the tracking error of tracking the moving target, and then transmit the error to the fuzzy control system, and according to the tracking error of the moving target and the error change rate EC, adjust the pitch angle adjustment value , horizontal rotation angle adjustment value and up and down movement distance adjustment value of the camera, so as to adjust the position and speed of tracking the moving target again, so as to track the target object.

[0086] Calculate the farthest recognition distance D of capturing moving objects or people, and the specific formula is as follows:

[0087]

[0088] In the formula, D max is the maximum recognition distance, is the focal length of the lens, is the object height, is the pixel size.

[0089] Embodiment 2

[0090] A multi-camera motion capture method based on fuzzy control and dynamic synchronization, comprising the following steps:

[0091] Step one: calculate the farthest recognition distance of multiple multi-cameras, and place the multiple multi-cameras within the farthest recognition distance of the moving object or person to be captured;

[0092] Step two: when the object or person starts to move, the multi-camera captures the moving image of the moving object or person, uploads the image to the fuzzy control system, and calculates the pitch angle adjustment value , horizontal rotation angle adjustment value and up and down movement distance adjustment value according to the position deviation and speed deviation of tracking the target object of the fuzzy control system, and the controller installed in the multi-camera adjusts the pitch angle adjustment value , horizontal rotation angle adjustment value and up and down movement distance adjustment value calculated by the fuzzy control system, so as to adjust the position and speed of tracking the moving target, so as to track the target object;

[0093] Step three: according to the position and speed of the moving target, calculate the tracking error of tracking the moving target, and then transmit the error to the fuzzy control system, and according to the tracking error of the moving target and the error change rate EC, adjust the pitch angle adjustment value , horizontal rotation angle adjustment value and up and down movement distance adjustment value , so as to adjust the position and speed of tracking the moving target again.

[0094] The farthest recognition distance D of the moving person or object is calculated, including the following steps:

[0095] Step a: obtaining the minimum recognition distance d1;

[0096] Step b: obtaining the maximum target surface width L2;

[0097] Step c: obtaining the single-pixel resolution ;

[0098] Step d: obtaining the farthest recognition distance D according to the obtained minimum recognition distance d1, maximum target surface width L2 and single-pixel resolution ;

[0099] In step a, the specific formula for obtaining the minimum recognition distance d1 is:

[0100] ;

[0101] In the formula, L1 is the camera target surface width, and θ is the lens divergence angle;

[0102] In step b, the specific formula for obtaining the maximum target surface width L2 is:

[0103] ;

[0104] In the formula, L1 is the camera target surface width, d1 is the minimum recognition distance, L2 is the maximum target surface width, is the incremental distance;

[0105] In step c, the specific formula for calculating the single-pixel resolution is:

[0106] ;

[0107] In the formula, R is the horizontal resolution of the camera, and L2 is the maximum target surface width;

[0108] In step d, the specific formula for calculating the farthest recognition distance D is:

[0109]

[0110] In the formula, L1 is the camera target surface width, is the lens divergence angle, P is the minimum recognition pixel, is the face transverse size, and R is the horizontal resolution of the camera.

[0111] Example 3

[0112] Based on Example 1 or Example 2, the pitch angle adjustment value in Step 2 is calculated , the horizontal rotation angle adjustment value and the up-down movement distance adjustment value The specific steps are as follows:

[0113] Step a: fuzzification processing is performed on the position deviation of the target object and the speed deviation of the target object to convert the accurate input value into a fuzzy value, and fuzzy inference is performed based on the fuzzy rule to calculate the fuzzy output value;

[0114] Step b: defuzzification is performed on the calculated fuzzy output value to convert it into an accurate pitch angle adjustment value , horizontal rotation angle adjustment value and up-down movement distance adjustment value The specific solving formula of the pitch angle adjustment value is as follows:

[0115]

[0116] The specific solving formula of the horizontal rotation angle adjustment value is as follows:

[0117]

[0118] The specific solving formula of the up-down movement distance adjustment value is as follows:

[0119]

[0120] In the above formula, is the membership degree of the fuzzy inference result, is the specific value of the fuzzy output variable, and i is the number of the inference rule. The fuzzy rule is specifically shown in Table 2;

[0121] Table 2 is the rule number and fuzzy rule definition

[0122]

[0123] In Step a, the process of fuzzification processing is to obtain the membership values of different fuzzy sets from the membership degree function according to the position deviation of the target object and the speed deviation of the target object , wherein the fuzzy sets include: negative large (NB), negative medium (NM), negative small (NS), zero (ZE), positive small (PS), positive medium (PM), and positive large (PB).

[0124] As the position deviation of the target object is inputted in step a and the velocity deviation of the target object , fuzzy processing is performed as shown in Table 1.

[0125] Table 1 is the definition of fuzzy sets and membership functions

[0126]

[0127] The fuzzy control system in step two solves the pitch angle adjustment value , the horizontal rotation angle adjustment value and the up and down movement distance adjustment value It can also be the following specific steps: six fuzzy variables are used, including adjusting the up pitch angle, the down pitch angle, the left horizontal rotation angle, the right horizontal rotation angle, the up movement distance and the down movement distance; each fuzzy variable is divided into a plurality of fuzzy subsets;

[0128] Fuzzy subsets of fuzzy variables (m=64 n=6 in the following formula):

[0129] if is and is … and is then

[0130] = + +…+ …

[0131] if is and is … and is then

[0132] = + +…+

[0133] ,… +… } as a parameter set

[0134] = =… =

[0135] = =… =

[0136] Parameter set can be simplified , … }where, identifies the first fuzzy inference rule, represents the mth fuzzy inference rule; represents the input variable of the system, the value of the actual system state; u 1 , 2 , m is the output result of the fuzzy inference rule, each u i is the output calculated according to the corresponding rule R m ; These are constant term parameters in each inference rule, which are not related to the input variable; is the corresponding coefficient of each input variable x1,x2,...,x n , Because of the equality relationship of , the coefficient Simplified parameter set, which represents the unified weight of each input variable in all rules; unify the coefficients of the same input variable in different fuzzy rules into a parameter set, simplify the parameter structure of the system, reduce the model complexity, make the parameter adjustment and system optimization become more simple and intuitive; The membership function of the fuzzy control set is to map the input variable to the fuzzy set, quantify the uncertainty, and provide the basis for fuzzy reasoning and decision-making; Two fuzzy subsets control the target; Each fuzzy variable uses three parameters to determine the membership function of the "positive" or "negative" fuzzy control set, which is used to adjust and optimize the tracking of the camera, and the specific formula is as follows:

[0137]

[0138] In the formula, represents the input variable, For the membership degree of the fuzzy set , the membership degree ranges from 0 to 1, indicating the membership degree of the variable in the fuzzy set ; is the actual value of the input variable; is the center position of the fuzzy set "positive", which determines the input value corresponding to the highest membership degree of the membership function; The center position of the fuzzy set "negative"; The extension parameter in the fuzzy set "positive" determines the "width" or "expansion" of the membership function, thus controlling the range of the fuzzy set; This is an extended parameter for the "negative" fuzzy set, used for the "negative" fuzzy set; The shape parameter of the fuzzy set "positive" controls the shape and rate of change of the membership function; The shape parameter for the "negative" fuzzy set, used for the "negative" fuzzy set.

[0139] Adjusting the pitch angle value in the fuzzy control system Horizontal rotation angle adjustment value and vertical movement distance adjustment value Input variables to fuzzy sets The output variable is determined to be the pitch angle adjustment value through fuzzy inference. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The fuzzy control rule is to use the fuzzy set of input variables. Convert to the corresponding value, pitch angle adjustment value Horizontal rotation angle adjustment value and vertical movement distance adjustment value The 49 values ​​of the fuzzy control rules are stored separately, and a lookup table is performed using base address and shift. Then, the pitch angle adjustment value is obtained by defuzzification using fuzzy inference. Horizontal rotation angle adjustment value and vertical movement distance adjustment value In the fuzzy set, NB represents negative large, NM represents negative medium, NS represents negative small, Z represents zero, PS represents positive small, PM represents positive medium, and PB represents positive large.

[0140] The content related to fuzzy control systems described above is existing technology in this field, and the above solution methods are only some of the existing solution methods.

[0141] In step two, the fuzzy control system will calculate the pitch angle adjustment value. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The pulse signal is converted into a pulse signal for transmitting a signal to the multi-view camera, adjusting the direction and focal length of the camera; the controller in the camera receives the pulse signal, and the camera performs corresponding adjustment according to the pulse signal; each pulse signal corresponds to a step, and the accumulated step angle adjusts the view angle and focal length of the camera; the controller is a stepping motor, and after the camera receives the pulse signal, the stepping motor controls the camera to rotate by a corresponding angle according to the signal frequency, signal quantity and signal strength, and changes the direction or focal length of the camera; the signal strength is used to control the rotation degree of the stepping motor to prevent over-rotation or insufficient rotation; the pulse signal contains control information and a synchronization identifier; the control information content includes timestamp information, which is used to record the generation time of the pulse signal and ensure the synchronization between multiple cameras; the synchronization identifier is used to ensure that each camera adjusts at the same time point to ensure the consistency of the captured action; the synchronization identifier is special information embedded in the pulse signal, and its main functions are to unify the time, coordinate the actions of multiple cameras, prevent signal interference and conflict, and accurately align the time; the pulse signal is sent by a pulse high-stability synchronization clock source installed on a Beidou chip or a GPS chip in each camera, and a unified synchronization clock signal is provided for multiple multi-view cameras.

[0142] Embodiment 4

[0143] Based on the basis of embodiment 3, the pitch angle adjustment value , the horizontal rotation angle adjustment value and the up-down movement distance adjustment value are obtained for adjusting and optimizing the tracking of the camera, and the position to be moved by the optimized camera is compared with the actual position to obtain the tracking error and the error change rate of the fuzzy control system, and the pitch angle adjustment value , the horizontal rotation angle adjustment value and the up-down movement distance adjustment value are adjusted by the fuzzy control system, and the actual position to be moved is obtained according to the obtained actual position to be moved, and the camera tracking is adjusted again during the movement.

[0144] The tracking error is calculated by a moving target tracking algorithm, and the actual position of the target at time t is , the predicted position of the target at time t is , and the specific formula of the tracking error is as follows:

[0145]

[0146] In the formula, P t is the actual position vector of the target at time t, which is usually represented by three-dimensional coordinates ; is the predicted position vector of the target at time t, usually represented as a three-dimensional coordinate ; is the tracking error vector at time t, represented as wherein: , , , wherein is the tracking error of the target in the x-axis, is the tracking error of the target in the y-axis, is the tracking error of the target in the z-axis, is the actual position of the target in the x-axis at time t, is the actual position of the target in the y-axis at time t, is the actual position of the target in the z-axis at time t, is the predicted position of the target in the x-axis at time t, is the predicted position of the target in the y-axis at time t, is the predicted position of the target in the z-axis at time t.

[0147] The actual position of the target is obtained from the captured images of multiple multi-view cameras through image processing algorithms , and the position of the target at time t is predicted using a dynamic model based on the previous motion trajectory and speed information of the target The specific formula for the prediction implementation is as follows:

[0148]

[0149] wherein, is the actual position of the target at time t-1, is the speed vector of the target at time t-1, represented as , is the time interval t-(t-1);

[0150] The error change rate EC is the change rate of the tracking error over time, and the specific formula is as follows:

[0151]

[0152] wherein, EC represents the speed of error change, and the size of the tracking error and the change direction of the error change rate EC are used to adjust the size of the pitch angle adjustment value , the horizontal rotation angle adjustment value , and the up-down movement distance adjustment value .

[0153] Embodiment 5

[0154] Based on the basis of example 4, the pulse high stable synchronous clock source receives the pulse source sent by Beidou and GPS respectively, the pulse source emits signals through local area network such as WIFI / LORA, the mean and variance of the count value of the pulse source sent by Beidou and GPS in the same time are calculated, the variance of the pulse source sent by Beidou and GPS is compared, the quality of Beidou and GPS signals is evaluated through the signal quality evaluation algorithm, the output time information of the chip is adjusted, the optimal pulse source is found out to assist the switching of the signal source, and the pulse signal is output; the pulse high stable synchronous clock source sends the pulse signal to a plurality of multi-view cameras through a wireless mode, the signal conversion module is arranged in the plurality of multi-view cameras, the pulse signal is converted into a control instruction of a stepping motor, and the control instruction controls the rotating direction, speed and rotating angle of the stepping motor; the time when the pulse signal is generated is determined through the signal quality evaluation algorithm, and when the signal quality Q exceeds the set threshold value, the pulse signal is generated.

[0155] After the pulse signal is generated, the local timestamp and the remote timestamp received in the synchronization cycle are collected first, the time offset is calculated through the timestamp synchronization algorithm , and finally the original timestamp received by each frame is adjusted , the adjusted timestamp of each frame , so that the frames captured by the cameras are aligned in time, and the multiple multi-view cameras are synchronized to track and capture moving objects in real time;

[0156] The local timestamp and the remote timestamp received in the synchronization cycle are collected, the time offset is calculated through the timestamp synchronization algorithm , and the specific formula for aligning the frames captured by the cameras in time is as follows:

[0157]

[0158] In the formula, is the adjusted timestamp, and the time alignment is ensured; is the original timestamp received; is the time offset;

[0159] The specific solving formula of the time offset is as follows:

[0160]

[0161] In the formula, is the local timestamp received for the i-th time; is the remote timestamp received for the i-th time, is the number of times of receiving the timestamp; the signal quality evaluation algorithm is used to evaluate the quality of Beidou and GPS signals, and the pulse signal is provided according to the signal quality Q, and the specific formula of the signal quality evaluation algorithm is as follows:

[0162]

[0163] In the formula, This is a signal quality evaluation value used to judge the quality of a signal; the higher the value, the better the signal quality. This is a weighting coefficient for the signal-to-noise ratio (SNR), typically ranging from 0 to 1, used to adjust the degree of influence of the SNR on the overall signal quality. This is a weighting coefficient for signal strength, typically ranging from 0 to 1, used to adjust the degree of influence of signal strength on overall signal quality. is the weighting coefficient for multipath interference, typically ranging from 0 to 1, used to adjust the negative impact of multipath interference on the overall signal quality; SS is the signal strength, representing the power intensity of the received signal. This is the multipath interference coefficient, used to quantify the degree of signal interference caused by multipath effects; Signal-to-noise ratio (SNR) is the ratio of signal strength to noise, usually expressed in decibels. A higher SNR indicates better signal quality. SNR and signal strength are important indicators of signal quality, measuring the clarity of a signal relative to background noise, while signal strength directly affects signal receptivity.

[0164] Calculate the signal-to-noise ratio The specific formula is:

[0165]

[0166] In the formula, For signal power, Noise power;

[0167] The time it takes for a signal to travel from the satellite to the receiver is used to determine the length of the signal transmission path. The specific formula for the signal arrival time is:

[0168]

[0169] In the formula, For signal arrival time, The time for receiving the signal. The time for sending the signal;

[0170] The clock offset is calculated to determine the difference between the receiver's internal clock and the satellite clock. The specific formula is as follows:

[0171]

[0172] In the formula, Due to clock skew, For receiver time, For satellite time;

[0173] The multi-path effect measure is obtained for judging the distortion or delay of the signal caused by the multiple paths to the receiver, and the specific formula is as follows:

[0174]

[0175] In the formula, is the multi-path effect measure, is the amplitude of the i-th path, is the amplitude of the i-th path, is the number of paths;

[0176] The signal delay is obtained for judging the time delay from the satellite sending to the receiver receiving, and the specific formula is as follows:

[0177]

[0178] In the formula, is the signal delay, is the signal transmission distance, is the speed of light, is other delay factors;

[0179] According to the results of the signal-to-noise ratio, the signal arrival time, the signal delay, the clock deviation and the multi-path effect measure obtained above, it is judged to select the signal source of Beidou or GPS, so as to output the optimal pulse source.

[0180] The signal quality evaluation algorithm is used to evaluate the quality of Beidou and GPS signals, and provides a pulse signal according to the signal quality Q. The specific formula of the signal quality evaluation algorithm is as follows:

[0181]

[0182] In the formula, Q is the signal quality, indicating the reliability of the signal; SS is the signal strength, indicating the received signal power strength; is the noise intensity, indicating the received background noise power.

[0183] The signal quality evaluation algorithm ensures the use of high-quality time signal source, ensuring the reliability of time synchronization; the dynamic synchronization algorithm of time stamp is used to dynamically adjust the time stamp by calculating the time offset, ensuring the time alignment of each camera and improving the capture accuracy; Beidou and GPS chips are deployed in the multi-camera system at the same time, enhancing the signal receiving ability and anti-interference ability of the system, improving the reliability and accuracy of time synchronization, which can effectively solve the time synchronization problem in the multi-camera system and ensure the time alignment of the multi-camera capture frame.

[0184] The above description is detailed for the preferred embodiments of the present application, but the embodiments are not intended to limit the scope of the patent application of the present application. Any equivalent changes or modifications made under the technical spirit of the present application should belong to the patent scope of the present application.

Claims

1. A multi-camera motion capture method based on fuzzy control and dynamic synchronization, characterized in that, Includes the following steps: Step 1: Calculate the farthest recognition distance of multiple multi-view cameras, and place multiple multi-view cameras within the farthest recognition distance of the moving object or person to be captured; Calculate the furthest recognition distance D for capturing moving people or objects, including the following steps: Step a: Obtain the minimum recognition distance d1; Step b: Obtain the maximum target width L2; Step c: Obtain single pixel resolution ; Step d: Based on the obtained minimum recognition distance d1, maximum target width L2, and single pixel resolution The furthest recognition distance D is obtained; In step a, the specific formula for obtaining the minimum recognition distance d1 is as follows: ； In the formula, L1 is the width of the camera target surface, and θ is the lens divergence angle; In step b, the specific formula for obtaining the maximum target width L2 is as follows: ； In the formula, L1 is the width of the camera target surface, d1 is the minimum recognition distance, and L2 is the maximum target surface width. Incremental distance; In step c, the single-pixel resolution is calculated. The specific formula is: ； In the formula, R is the horizontal resolution of the camera, and L2 is the maximum target width; The specific formula for calculating the farthest recognition distance D in step d is as follows: ； In the formula, L1 is the width of the camera target surface. Where P is the lens divergence angle and P is the minimum recognizable pixel. R represents the horizontal dimension of the face, and R represents the horizontal resolution of the camera. Step Two: When an object or person begins to move, the multi-view camera captures the motion image of the moving object or person and uploads the image to the fuzzy control system. The fuzzy control system then tracks the position deviation of the target object. and the velocity deviation of the target object The pitch angle adjustment value was calculated. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The controller installed inside the multi-view camera adjusts the pitch angle based on the value calculated by the fuzzy control system. Horizontal rotation angle adjustment value and vertical movement distance adjustment value Adjust the position and speed of the moving target to make it track the target object; Calculate the pitch angle adjustment value in step two. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The specific steps are as follows: Step a: Deviate the position of the target object velocity deviation from the target object The precise input value is converted into a fuzzy value, and fuzzy inference is performed based on fuzzy rules to calculate the fuzzy output value. Step b: Defuzzify the calculated blurry output values ​​and convert them into precise pitch angle adjustment values. Horizontal rotation angle adjustment value and vertical movement distance adjustment value ; In step a, the fuzzing process involves adjusting the positional deviation of the target object. velocity deviation from the target object The membership values ​​of different fuzzy sets are obtained according to the membership function, where the fuzzy sets include: negative large, negative medium, negative small, zero, positive small, positive medium, and positive large. In step two, the fuzzy control system will calculate the pitch angle adjustment value. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The signals are converted into pulse signals, which are used to transmit signals to the cameras to adjust their direction and focus. The controller inside the camera receives the pulse signals and performs corresponding adjustments. Each pulse signal corresponds to a step, and these step angles are accumulated to adjust the camera's viewing angle and focus. The controller is a stepper motor. After receiving the pulse signals, the stepper motor controls the camera to rotate by a corresponding angle and change its orientation or focus based on the signal frequency, number of signals, and signal strength. The signal strength controls the rotation force of the stepper motor to prevent over-rotation or under-rotation. The pulse signals contain control information and synchronization identifiers. The control information includes timestamps to record the generation time of the pulse signals, ensuring synchronization between multiple cameras. The synchronization identifier ensures that each camera adjusts at the same time, guaranteeing consistent capture actions. The pulse signals are transmitted using a high-stability pulse synchronization clock source installed on the Beidou or GPS chip in each camera, providing a unified synchronization clock signal for multiple multi-view cameras. The pulse high-stability synchronous clock source receives pulses from BeiDou and GPS respectively, calculates the mean and variance of the count values ​​of the received pulses from BeiDou and GPS within the same time period, compares the variances of the pulses from BeiDou and GPS, evaluates the quality of BeiDou and GPS signals through a signal quality assessment algorithm, adjusts the chip's output timing information, finds the optimal pulse source to assist in signal source switching, and outputs a pulse signal. The pulse high-stability synchronous clock source wirelessly transmits the pulse signal to multiple multi-view cameras. The multiple multi-view cameras have internal signal conversion modules that convert the pulse signal into control commands for stepper motors. The control commands control the rotation direction, speed, and rotation angle of the stepper motors. The pulse signal generation time is determined by the signal quality assessment algorithm. When the signal quality Q exceeds a set threshold, a pulse signal is generated. After the pulse signal is generated, the local and remote timestamps received within the synchronization period are first collected. Then, the time offset is calculated using a timestamp synchronization algorithm. Finally, adjust the original timestamps received for each frame. , is the adjusted timestamp for each frame. This ensures that the frames captured by the camera are aligned in time, enabling multiple cameras to simultaneously track and capture moving objects in real time. Collect local and remote timestamps received during the synchronization period, and calculate the time offset using a timestamp synchronization algorithm. The specific formula for aligning the frames captured by each camera in time is as follows: ； In the formula, For the adjusted timestamps, ensure time alignment; The received original timestamp; This is the time offset; The specific formula for calculating the time offset is as follows: ； In the formula, Let i be the local timestamp of the i-th received data. Let i be the remote timestamp received for the i-th time. The number of timestamps received; The signal quality assessment algorithm is used to evaluate the quality of BeiDou and GPS signals, providing pulse signals based on the signal quality Q. The specific formula of the signal quality assessment algorithm is as follows: Mode as follows: ； In the formula, This is a signal quality evaluation value used to judge the quality of a signal; the higher the value, the better the signal quality. This is a weighting coefficient for the signal-to-noise ratio (SNR), typically ranging from 0 to 1, used to adjust the degree of influence of the SNR on the overall signal quality. This is a weighting coefficient for signal strength, typically ranging from 0 to 1, used to adjust the degree of influence of signal strength on overall signal quality. is the weighting coefficient for multipath interference, typically ranging from 0 to 1, used to adjust the negative impact of multipath interference on the overall signal quality; SS is the signal strength, representing the power intensity of the received signal. This is the multipath interference coefficient, used to quantify the degree of signal interference caused by multipath effects; Signal-to-noise ratio (SNR) is the ratio of signal strength to noise, usually expressed in decibels. A higher SNR indicates better signal quality. Calculate the signal-to-noise ratio The specific formula is: ； In the formula, For signal power, Noise power; The time it takes for a signal to travel from the satellite to the receiver is used to determine the length of the signal transmission path. The specific formula for the signal arrival time is: ； In the formula, For signal arrival time, The time for receiving the signal. The time for sending the signal; The clock offset is calculated to determine the difference between the receiver's internal clock and the satellite clock. The specific formula is as follows: ； In the formula, Due to clock skew, For receiver time, For satellite time; The multipath effect metric is calculated to determine the distortion or delay of the signal caused by multiple paths to the receiver. The specific formula is as follows: ； In the formula, For measuring multipath effects, Let be the amplitude of the i-th path. Let be the amplitude of the i-th path. This represents the number of paths. The signal delay is calculated to determine the time delay from satellite transmission to receiver reception. The specific formula is as follows: ； In the formula, For signal delay, For signal transmission distance, At the speed of light, Other delaying factors; Based on the results of the signal-to-noise ratio, signal arrival time, signal delay, clock skew, and multipath effect measurement obtained above, it is determined whether to select the signal source from BeiDou or GPS, thereby outputting the optimal pulse source. Step 3: Based on the position and speed of the moving target, calculate the tracking error, and then transmit the error to the fuzzy control system. The fuzzy control system then uses the tracking error to track the moving target. And the error change rate EC, adjust the camera's pitch angle adjustment value. Horizontal rotation angle adjustment value and vertical movement distance adjustment value This allows the position and speed of the moving target to be readjusted so that it can track the target object.

2. The multi-camera motion capture method based on fuzzy control and dynamic synchronization according to claim 1, characterized in that: The pitch angle adjustment value will be obtained. Horizontal rotation angle adjustment value and vertical movement distance adjustment value The tracking of the camera is adjusted and optimized, and then the position that the optimized camera needs to move to is compared with the actual position to obtain the tracking error of the fuzzy control system. The pitch angle adjustment value is adjusted by the fuzzy control system based on the error change rate EC. Horizontal rotation angle adjustment value and vertical movement distance adjustment value During the movement, the camera tracking is readjusted based on the actual position to be moved. Tracking error The moving target tracking algorithm is used for calculation. Let the actual position of the target at time t be... The predicted position of the target at time t is Then the tracking error The specific formula is as follows: ； In the formula, P t Let be the actual position vector of the target at time t, usually represented as three-dimensional coordinates. ; Let be the predicted position vector of the target at time t, usually represented as three-dimensional coordinates. ; Let be the tracking error vector at time t, denoted as: ,in: , , In the formula The tracking error of the target on the x-axis. The tracking error of the target on the y-axis, The tracking error of the target along the z-axis. Let t be the actual position of the target on the x-axis at time t. Let be the actual position of the target on the y-axis at time t. Let be the actual position of the target on the z-axis at time t. The predicted position of the target on the x-axis at time t. Let y be the predicted position of the target on the y-axis at time t. The predicted position of the target on the z-axis at time t; The actual position of the target is obtained from images captured by multiple multi-view cameras through image processing algorithms. Based on the target's previous trajectory and velocity information, a dynamic model is used to predict the target's position at time t. The specific formula for the prediction is as follows: ； In the formula, The actual position of the target at time t-1, Let be the velocity vector of the target at time t-1, denoted as: , The time interval is t-(t-1); The rate of change of error EC is the tracking error. The rate of change over time is given by the following formula: ； In the formula, EC represents the rate of change of the error, which is determined based on the tracking error. The pitch angle adjustment value is adjusted based on the magnitude and direction of the error rate of change EC. Horizontal rotation angle adjustment value and vertical movement distance adjustment value Size.

Images