Visual speed measurement method, device, equipment and storage medium

Abstract

The present invention discloses a visual speed measurement method, device, equipment and storage medium, the method comprising: determining the movement speed of the camera based on the linear acceleration of the camera measured by an inertial measurement unit and the result of the posture solution of the camera; determining the compensation displacement of the camera according to a preset duration and the movement speed; determining the detection displacement of the target to be detected in the video shot by the camera during the preset duration; determining the actual displacement according to the compensation displacement and the detection displacement, and determining the actual speed of the target to be detected according to the preset duration and the actual displacement. The present invention improves the accuracy of the actual displacement and the actual speed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a visual speed measurement method, device, equipment and storage medium. Background Art

[0002] In terms of speed detection of moving targets, speed measuring devices such as radars or cameras are mainly used to detect the movement of moving targets in a specific speed measurement area within the camera field of view of the speed measuring device, thereby detecting the speed of the moving target. For example, to detect the speed of vehicles on the road, the speed measuring device often needs to be fixed at a specific location to detect the movement of moving targets in a specific speed measurement area, thereby performing speed detection. However, in competitive sports, when tracking a moving target, the camera of the speed measuring device will move with the camera, resulting in inaccurate speed detection of the moving target. Summary of the invention

[0003] The main purpose of the present invention is to provide a visual speed measurement method, device, equipment and storage medium, aiming to solve the problem of inaccurate speed detection of moving targets in a camera.

[0004] To achieve the above object, the present invention provides a visual speed measurement method, which comprises the following steps:

[0005] Determine the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture solution of the camera;

[0006] Determining the compensation displacement of the camera according to a preset duration and the movement speed;

[0007] Determining a detection displacement of a target to be detected in the video captured by the camera during the preset time period;

[0008] The actual displacement is determined according to the compensation displacement and the detection displacement, and the actual speed of the target to be detected is determined according to the preset time length and the actual displacement.

[0009] In one embodiment, before the step of determining the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture calculation of the camera, the step further includes:

[0010] Get the linear acceleration and angular acceleration of the inertial measurement unit;

[0011] Determine a linear velocity and an angular velocity of the camera according to the linear acceleration and the angular acceleration;

[0012] The attitude parameters of the camera are determined according to the linear velocity and the angular velocity, and the attitude solution result includes the attitude parameters.

[0013] In one embodiment, the step of determining the linear velocity and the angular velocity of the camera according to the linear acceleration and the angular acceleration comprises:

[0014] Performing an integration operation on the linear acceleration within a preset time period to obtain the linear velocity;

[0015] The angular velocity is obtained by performing an integration operation on the angular acceleration within a preset time period.

[0016] In one embodiment, the step of determining the posture parameter of the camera according to the linear velocity and the angular velocity comprises:

[0017] Determine the gravitational acceleration of the camera according to the angular velocity;

[0018] Determine a gravity error value according to the normalized linear acceleration and the gravity acceleration;

[0019] Determine a corrected gravity acceleration according to a preset proportionality coefficient, the gravity acceleration and the gravity error value;

[0020] The attitude parameter is determined according to the angular velocity and the corrected gravitational acceleration.

[0021] In one embodiment, the step of determining the gravity error value according to the normalized linear acceleration and the gravity acceleration comprises:

[0022] Cross-multiply the normalized linear acceleration and the gravitational acceleration to calculate a target error;

[0023] An integration operation is performed according to the target error within a preset time period to obtain the gravity error value.

[0024] In one embodiment, the step of determining the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture calculation of the camera includes:

[0025] Determining a target matrix of the posture parameters;

[0026] determining the gravitational acceleration of the camera according to the angular velocity;

[0027] The movement speed of the camera is determined according to the target matrix, the gravitational acceleration and the linear acceleration.

[0028] In one embodiment, the step of determining the preset time duration, wherein the step of detecting the displacement of the target to be detected in the video captured by the camera comprises:

[0029] Determine the actual length of the target structure of the target to be detected in the video captured by the camera, and the pixel length of the target structure;

[0030] determining a scale factor according to the actual distance and the pixel length;

[0031] Determine the pixel distance moved by the target to be detected within a preset time period;

[0032] A detection displacement corresponding to the target to be detected is determined according to the scale factor and the pixel distance.

[0033] To achieve the above object, a visual speed measuring device is provided, the visual speed measuring device comprising:

[0034] A determination module, configured to determine a movement speed of the camera based on a linear acceleration of the camera measured by an inertial measurement unit and a result of solving a posture of the camera;

[0035] A first calculation module, used to determine the compensation displacement of the camera according to a preset duration and the movement speed;

[0036] A second calculation module is used to determine the detection displacement of the target to be detected in the video captured by the camera during the preset time period;

[0037] The speed measurement module is used to determine the actual displacement according to the compensation displacement and the detection displacement, and to determine the actual speed of the target to be detected according to the preset time and the actual displacement.

[0038] To achieve the above objectives, the present invention also provides a visual speed measurement device, which includes a memory, a processor, and a visual speed measurement program stored in the memory and executable on the processor, and when the visual speed measurement program is executed by the processor, it implements the various steps of the visual speed measurement method described above.

[0039] To achieve the above objectives, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a visual speed measurement program, and when the visual speed measurement program is executed by a processor, the various steps of the visual speed measurement method described above are implemented.

[0040] The present invention provides a visual speed measurement method, device, equipment and storage medium, which determines the movement speed of the camera based on the linear acceleration of the camera measured by an inertial measurement unit and the result of the camera's posture solution; determines the compensation displacement according to the preset duration and the movement speed; determines the detection displacement of the target to be detected in the video shot by the camera during the preset duration; determines the actual displacement according to the compensation displacement and the detection displacement, and determines the actual speed of the target to be detected according to the preset duration and the actual displacement. By determining the compensation displacement of the camera and the detection displacement of the target to be detected, and determining the actual displacement and actual speed of the target to be detected according to the compensation displacement and the detection displacement, the speed measurement error caused by the camera movement is avoided, and the accuracy of the actual displacement and actual speed of the target to be detected during the actual movement process is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the hardware structure of the visual speed measurement device involved in the embodiment of the present invention;

[0042] Figure 2 It is a flowchart of a first embodiment of the visual speed measurement method of the present invention;

[0043] Figure 3 It is a flow chart of a second embodiment of the visual speed measurement method of the present invention;

[0044] Figure 4 Schematic diagram of a detailed flow chart of step S30 of the third embodiment of the visual speed measurement method of the present invention;

[0045] Figure 5 It is a schematic diagram of the logical structure of the visual speed measurement device of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] The main solution of the embodiment of the present invention is: determine the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the posture solution result of the camera; determine the compensation displacement according to the preset time length and the movement speed; determine the detection displacement of the target to be detected in the video shot by the camera during the preset time length; determine the actual displacement according to the compensation displacement and the detection displacement, and determine the actual speed of the target to be detected according to the preset time length and the actual displacement.

[0049] By determining the compensation displacement of the camera and the detection displacement of the target to be detected, and determining the actual displacement and actual speed of the target to be detected based on the compensation displacement and the detection displacement, the speed measurement error caused by the camera movement is avoided, and the accuracy of the actual displacement and actual speed of the target to be detected during the actual movement process is improved.

[0050] As an implementation solution, visual speed measurement equipment can be Figure 1 shown.

[0051] The embodiment of the present invention relates to a visual speed measurement device, which includes: a processor 101, such as a CPU, a memory 102, and a communication bus 103. The communication bus 103 is used to realize connection and communication between these components.

[0052] The memory 102 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Figure 1 As shown, the memory 102 as a computer-readable storage medium may include a visual speed measurement program; and the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0053] Determine the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture solution of the camera;

[0054] Determining the compensation displacement of the camera according to a preset duration and the movement speed;

[0055] Determining a detection displacement of a target to be detected in the video captured by the camera during the preset time period;

[0056] The actual displacement is determined according to the compensation displacement and the detection displacement, and the actual speed of the target to be detected is determined according to the preset time length and the actual displacement.

[0057] In one embodiment, the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0058] Get the linear acceleration and angular acceleration of the inertial measurement unit;

[0059] Determine a linear velocity and an angular velocity of the camera according to the linear acceleration and the angular acceleration;

[0060] The attitude parameters of the camera are determined according to the linear velocity and the angular velocity, and the attitude solution result includes the attitude parameters.

[0061] In one embodiment, the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0062] Performing an integration operation on the linear acceleration within a preset time period to obtain the linear velocity;

[0063] The angular velocity is obtained by performing an integration operation on the angular acceleration within a preset time period.

[0064] In one embodiment, the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0065] Determine the gravitational acceleration of the camera according to the angular velocity;

[0066] Determine a gravity error value according to the normalized linear acceleration and the gravity acceleration;

[0067] Determine a corrected gravity acceleration according to a preset proportionality coefficient, the gravity acceleration and the gravity error value;

[0068] The attitude parameter is determined according to the angular velocity and the corrected gravitational acceleration.

[0069] In one embodiment, the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0070] Cross-multiply the normalized linear acceleration and the gravitational acceleration to calculate a target error;

[0071] An integration operation is performed according to the target error within a preset time period to obtain the gravity error value.

[0072] In one embodiment, the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0073] Determining a target matrix of the posture parameters;

[0074] determining the gravitational acceleration of the camera according to the angular velocity;

[0075] The movement speed of the camera is determined according to the target matrix, the gravitational acceleration and the linear acceleration.

[0076] In one embodiment, the processor 101 may be used to call the visual speed measurement program stored in the memory 102 and perform the following operations:

[0077] Determine the actual length of the target structure of the target to be detected in the video captured by the camera, and the pixel length of the target structure;

[0078] determining a scale factor according to the actual distance and the pixel length;

[0079] Determine the pixel distance moved by the target to be detected within a preset time period;

[0080] A detection displacement corresponding to the target to be detected is determined according to the scale factor and the pixel distance.

[0081] Based on the hardware architecture of the above-mentioned visual speed measurement device, an embodiment of the visual speed measurement method of the present invention is proposed.

[0082] Reference Figure 2 , Figure 2 This is the first embodiment of the visual speed measurement method of the present invention, and the visual speed measurement method comprises the following steps:

[0083] Step S10, determining the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture calculation of the camera;

[0084] Specifically, the camera is provided with an IMU (Inertial Measurement Unit). When the camera shoots a moving target, due to the movement of the camera itself, the moving distance of the moving target obtained only by visual measurement has an offset. The motion compensation of the camera can be performed with the help of the inertial measurement unit, wherein the inertial measurement unit is composed of an accelerometer and a gyroscope, the accelerometer is used to measure linear acceleration, and the gyroscope is used to measure angular acceleration. The attitude solution result of the camera is obtained according to the linear acceleration and angular acceleration of the inertial measurement unit, wherein the attitude solution result is the attitude parameter of the camera, and the attitude parameter can be represented by Euler angles, direction cosine matrix or quaternion. Since the camera lacks depth information, the compensation displacement can be determined by integrating the camera displacement of adjacent frames by increasing the video sampling rate.

[0085] Based on the attitude solution result of the inertial measurement unit and the linear acceleration corresponding to the camera, the movement speed of the camera is determined, wherein the attitude solution result includes a second attitude parameter. Exemplarily, the second attitude parameter is determined The reverse rotation cosine matrix R, that is, the target matrix, where R can be solved by The inverse q′ is used to calculate the cosine matrix; the linear acceleration a is converted to the geographic coordinate system, and the gravity acceleration g is subtracted, and the camera motion speed V is finally integrated. c,k , as shown below:

[0086] V c,k =V c,k-1 +(Ra-g)Δt;

[0087] Step S20, determining the compensation displacement of the camera according to the preset duration and the movement speed;

[0088] Specifically, the compensation displacement is the motion displacement of the camera, and the compensation displacement is determined according to a preset duration and motion speed. Optionally, the camera motion speed V c,k Perform time integration to obtain the camera displacement D c , exemplarily, as follows:

[0089] D c =∫V c,k Δt.

[0090] Step S30, determining the detection displacement of the target to be detected in the video captured by the camera during the preset time period;

[0091] Specifically, the target to be detected is a moving target photographed by a camera, or a moving target selected by a user; the detected displacement is the displacement d of the target to be detected photographed by the camera. s,k Determine the detection displacement corresponding to the detection target within the preset time length. Optionally, obtain a detection model by training a neural network model, input the video frames captured by the camera into the detection model, and obtain the detection displacement corresponding to the target to be detected within the preset time length.

[0092] Step S40, determining the actual displacement according to the compensation displacement and the detection displacement, and determining the actual speed of the target to be detected according to the preset time length and the actual displacement.

[0093] Specifically, the actual displacement is determined according to the detected displacement and the compensated displacement, and the actual speed of the target to be detected is determined according to the preset time and the actual displacement. Optionally, the actual displacement is determined according to the sum of the detected displacement and the compensated displacement, and the actual speed of the target to be detected is determined according to the ratio of the actual displacement to the preset time, as shown in the following formula:

[0094] P=D c +d s,k ;

[0095]

[0096] Where P is the actual displacement; V k is the actual speed; d s,k To detect displacement; D c To compensate for displacement.

[0097] Optionally, the displacement of the camera and the displacement of the athlete can be obtained by the rigid reference object at time k and k+1, i.e., the target structure (L i,k ,L i,k+1 ) as the scale factor to obtain the actual displacement P and actual speed V of the moving target.k , as shown in the following formula:

[0098] P=s′D c +d s,k ;

[0099]

[0100] Among them, d s,k To detect displacement; D c To compensate for displacement.

[0101] In the technical solution of this embodiment, the camera's movement speed is determined based on the camera's linear acceleration measured by the inertial measurement unit and the camera's posture solution result; the compensation displacement is determined according to the preset duration and the movement speed; the detection displacement of the target to be detected in the video shot by the camera during the preset duration is determined; the actual displacement is determined according to the compensation displacement and the detection displacement, and the actual speed of the target to be detected is determined according to the preset duration and the actual displacement. By determining the camera's compensation displacement and the detection displacement of the target to be detected, and determining the actual displacement and actual speed of the target to be detected according to the compensation displacement and the detection displacement, the speed measurement error caused by the camera movement is avoided, and the accuracy of the actual displacement and actual speed of the target to be detected during the actual movement process is improved.

[0102] Reference Figure 3 , Figure 3 This is the second embodiment of the visual speed measurement method of the present invention. Based on the first embodiment, before step S10, the method further includes:

[0103] Step S50, obtaining the linear acceleration and angular acceleration of the inertial measurement unit;

[0104] Step S60, determining the linear velocity and angular velocity of the camera according to the linear acceleration and the angular acceleration;

[0105] Step S70, determining the attitude parameters of the camera according to the linear velocity and the angular velocity, and the attitude solution result includes the attitude parameters.

[0106] Specifically, the linear acceleration (a) measured by the inertial measurement unit is obtained. x ,a y ,a z ) and angular acceleration (w x ,w y ,w z ), according to the linear acceleration and angular acceleration, the linear velocity and angular velocity of the camera are determined, and the linear velocity can be obtained by integrating the linear acceleration within a preset time length; and the angular velocity can be obtained by integrating the angular acceleration within a preset time length.

[0107] Linear acceleration (a x ,a y ,a z ) and angular acceleration (w x ,w y ,w z ) is a discrete observation value. Within the sampling period Δt of the video frame, the quaternion q of the linear velocity v and angular velocity a of the camera can be obtained by the discrete integration of the observation value. For example, the median method is used to solve it, as shown in the following formula:

[0108]

[0109]

[0110] in, and are the linear velocity and angular velocity at time k, a k+1 and w k+1 are the average linear acceleration and average angular acceleration in the time period Δt [k, k+1], respectively. and In the initial state k=0, it is set to [0,0,0] and [1,0,0,0].

[0111]

[0112]

[0113] Among them, a k+1 and w k+1 are the average linear acceleration and average angular acceleration in the time period Δt [k, k+1], respectively. are the linear acceleration and angular acceleration in the IMU coordinate system at time k and k+1 respectively, is the angular velocity drift of the gyroscope at time k, that is, the angular velocity bias, is the gravity drift of the gyroscope at time k, i.e., the gravity acceleration bias, which can be obtained in advance through offline calibration, g w is the acceleration due to gravity.

[0114] The accelerometer observations are normalized so that the value range of a is [0,1], as shown in the following formula:

[0115]

[0116]

[0117] Determine the gravitational acceleration of the camera according to the angular velocity, and calculate the initial matrix DCM through the equivalent transformation formula of the direction cosine matrix and the quaternion:

[0118]

[0119] Among them, the angular velocity is

[0120] The gravitational acceleration of the camera is determined according to the initial matrix. The direction of gravity is perpendicular to the center of the earth, that is, the gravity component is the z-axis component. Each row in the direction cosine matrix DCM represents a direction axis (x, y, z). The quaternion The gravity component g in the equivalent cosine matrix of i Separate the components of gravity on the three axes of the current posture, and the gravitational acceleration is as follows:

[0121] g ix =2(q1q3-q0q2);

[0122] g iy =2(q0q1-q2q3);

[0123] g iz =1-2(q1q1+q2q2).

[0124] The gravity error value is determined based on the normalized linear acceleration and gravity acceleration. Optionally, the normalized linear acceleration is cross-multiplied with the gravity acceleration to calculate the target error. The gravity error value is obtained by integrating the target error within a preset time. i Separate and perform cross product to calculate the gravity error V deltak , as shown below:

[0125]

[0126] Integrate the gravity error within the time Δt to obtain the gravity error value V delta .

[0127] The corrected gravity acceleration is determined according to the preset proportional coefficient, gravity acceleration and gravity error value. Optionally, the gravity error value V delta Add to the gravitational acceleration g i Perform error elimination on As shown in the following formula:

[0128]

[0129] in, Indicates the intermediate error, g i is the acceleration due to gravity, V delta Represents the gravity error value, K i Represents the proportionality coefficient, Ki This can be obtained by adjusting parameters experimentally before using the device. In actual scenarios, 0 can be used to simplify the error.

[0130] The intermediate error calculated above is proportionally calculated to obtain the corrected gravity acceleration g~ i ,

[0131]

[0132] in, represents the corrected gravitational acceleration; Indicates the middle error, V delta Represents the gravity error value, K p is the proportionality coefficient, K p This can be obtained by adjusting parameters experimentally before using the device. In actual scenarios, 0 can be used to simplify the error.

[0133] The attitude parameters are determined based on the angular velocity and the corrected gravity acceleration. Since the rotation matrix is ​​an orthogonal matrix, the corrected gravity acceleration The modulus is 1, which can be expressed as a quaternion. Finally, the IMU's attitude is corrected according to the gravity acceleration Perform rotation update to obtain the final posture parameters The rotation of the attitude quaternion can be obtained by cross-multiplying two quaternions, as shown in the following formula:

[0134]

[0135]

[0136]

[0137]

[0138] Optionally, the movement speed of the camera is determined based on the attitude solution result of the inertial measurement unit and the linear acceleration corresponding to the camera, and the target matrix of the attitude parameters is determined; illustratively, the attitude parameters are determined according to the attitude solution result The reverse rotation cosine matrix R, that is, the target matrix, where R can be solved by The inverse q′ of the cosine matrix is ​​calculated to obtain, for example, the target matrix R is as follows:

[0139]

[0140] Determine the camera's gravity acceleration based on the angular velocity, as described above, and will not be repeated here; determine the camera's movement speed based on the target matrix, gravity acceleration and linear acceleration, convert the linear acceleration a to the geographic coordinate system, and subtract the gravity acceleration g, and finally integrate to obtain the camera's movement speed V c,k . For example, the formula is as follows:

[0141] V c,k =V c,k-1 +(Ra-g)Δt;

[0142] Among them, V c,k represents the speed of the camera at time k, V c,k-1 It represents the speed of the camera at time k-1, R is the target matrix, a is the linear acceleration, and g is the gravitational acceleration.

[0143] In the technical solution of this embodiment, the gravity acceleration of the camera is determined according to the angular velocity; the gravity error value is determined according to the normalized linear acceleration and gravity acceleration; the corrected gravity acceleration is determined according to the preset proportional coefficient, gravity acceleration and gravity error value; the attitude parameter is determined according to the angular velocity and the corrected gravity acceleration. The attitude of the camera is solved by the inertial measurement unit to obtain the attitude solution result, so as to avoid the speed measurement error caused by the camera movement and improve the accuracy of the actual displacement and actual speed of the target to be detected during the actual movement process.

[0144] Reference Figure 4 , Figure 4 This is the third embodiment of the visual speed measurement method of the present invention, based on the first or second embodiment, step S30 includes:

[0145] Step S31, determining the actual length of the target structure of the target to be detected in the video captured by the camera, and the pixel length of the target structure;

[0146] Step S32, determining a scale factor according to the actual distance and the pixel length;

[0147] Step S33, determining the pixel distance moved by the target to be detected within a preset time period;

[0148] Step S34: determining a detection displacement corresponding to the target to be detected according to the scale factor and the pixel distance.

[0149] Specifically, in order to solve the problem of scale loss in visual positioning, the actual length of the target structure of the target to be detected in the video captured by the camera and the pixel length of the target structure are determined. During the movement process, the target to be detected is always in the field of view of the camera, and a rigid structure in the structure of the target to be detected can be selected as a scale reference, i.e., the target structure. Using the target detection algorithm, the pixel length Li can be obtained by detecting the endpoints P1 and P2 of the target structure, and then the scale factor is determined according to the actual distance Lp of the target structure and the actual distance and pixel length. For example, as shown in the following formula:

[0150]

[0151] Optionally, in the 100-meter sprint event, target detection algorithms such as the currently open source RetinaNet are used to detect athletes, and then human skeleton key points are detected on the target, such as the open source OpenPose. Track data is used to manually annotate skeleton key points to perform network fine-tuning. Since the human skeleton is a rigid structure, the upper arm / forearm, etc. can be selected as the target structure, and the actual length of the target structure can be determined.

[0152] Optionally, in the ski jumping event, the athlete is detected by a target detection algorithm, such as the currently open source RetinaNet, and then the snowboard is detected, such as the open source OpenPose, and the network is fine-tuned by manually annotating endpoints using the snowboard data. Since the snowboard is a rigid structure, the snowboard can be selected as the target structure and the actual length of the target structure can be determined.

[0153] Determine the pixel distance of the target to be detected moving within a preset time length; determine the detection displacement corresponding to the target to be detected based on the scale factor and the pixel distance. Optionally, determine the pixel distance of the target to be detected corresponding to different video frames of the video to be processed within a preset time length, and determine the detection displacement corresponding to the target to be detected based on the scale factor and each pixel distance. Exemplarily, the center point of the detection frame is used to simplify the target to be detected, and different pixel positions (x0, y0), (x1, y1), ..., (xi-1, yi-1), (xi, yi) are obtained in the previous and next frames. The pixel distance can be obtained by the pixel coordinate distance of adjacent frames. The final physical distance is obtained by multiplying the pixel distance by the scale factor:

[0154]

[0155] In the technical solution of this embodiment, the actual length of the target structure of the target to be detected in the video captured by the camera and the pixel length of the target structure are determined; the scale factor is determined according to the actual distance and the pixel length; the pixel distance moved by the target to be detected within a preset time length is determined; and the detection displacement corresponding to the target to be detected is determined according to the scale factor and the pixel distance. The detection displacement corresponding to the target to be detected in the captured video is accurately determined by the scale factor and the pixel distance, so as to accurately determine the actual displacement and actual speed of the target to be detected during the actual movement process.

[0156] Reference Figure 5 The present invention also provides a visual speed measuring device, the visual speed measuring device comprising:

[0157] A determination module 100, configured to determine a movement speed of the camera based on a linear acceleration of the camera measured by an inertial measurement unit and a result of solving a posture of the camera;

[0158] A first calculation module 200, used to determine the compensation displacement of the camera according to a preset duration and the movement speed;

[0159] A second calculation module 300 is used to determine the detection displacement of the target to be detected in the video captured by the camera during the preset time period;

[0160] The speed measurement module 400 is used to determine the actual displacement according to the compensation displacement and the detection displacement, and to determine the actual speed of the target to be detected according to the preset time and the actual displacement.

[0161] In one embodiment, before determining the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the attitude solution result of the camera, the determination module 100 is specifically used to:

[0162] Get the linear acceleration and angular acceleration of the inertial measurement unit;

[0163] Determine a linear velocity and an angular velocity of the camera according to the linear acceleration and the angular acceleration;

[0164] The attitude parameters of the camera are determined according to the linear velocity and the angular velocity, and the attitude solution result includes the attitude parameters.

[0165] In one embodiment, in determining the linear velocity and the angular velocity of the camera according to the linear acceleration and the angular acceleration, the determination module 100 is specifically configured to:

[0166] Performing an integration operation on the linear acceleration within a preset time period to obtain the linear velocity;

[0167] The angular velocity is obtained by performing an integration operation on the angular acceleration within a preset time period.

[0168] In one embodiment, in determining the posture parameter of the camera according to the linear velocity and the angular velocity, the determination module 100 is specifically used to:

[0169] Determine the gravitational acceleration of the camera according to the angular velocity;

[0170] Determine a gravity error value according to the normalized linear acceleration and the gravity acceleration;

[0171] Determine a corrected gravity acceleration according to a preset proportionality coefficient, the gravity acceleration and the gravity error value;

[0172] The attitude parameter is determined according to the angular velocity and the corrected gravitational acceleration.

[0173] In one embodiment, in determining the gravity error value according to the normalized linear acceleration and the gravity acceleration, the determination module 100 is specifically configured to:

[0174] Cross-multiply the normalized linear acceleration and the gravitational acceleration to calculate a target error;

[0175] An integration operation is performed according to the target error within a preset time period to obtain the gravity error value.

[0176] In one embodiment, in determining the movement speed of the camera based on the attitude calculation result of the inertial measurement unit and the linear acceleration corresponding to the camera, the determination module 100 is specifically used to:

[0177] Determining a target matrix of the posture parameters;

[0178] determining the gravitational acceleration of the camera according to the angular velocity;

[0179] The movement speed of the camera is determined according to the target matrix, the gravitational acceleration and the linear acceleration.

[0180] In one embodiment, in determining the preset time duration, in terms of the detection displacement of the target to be detected in the video captured by the camera, the first calculation module 200 is specifically used to:

[0181] Determine the actual length of the target structure of the target to be detected in the video captured by the camera, and the pixel length of the target structure;

[0182] determining a scale factor according to the actual distance and the pixel length;

[0183] Determine the pixel distance moved by the target to be detected within a preset time period;

[0184] A detection displacement corresponding to the target to be detected is determined according to the scale factor and the pixel distance.

[0185] The present invention also provides a visual speed measurement device, which includes a memory, a processor, and a visual speed measurement program stored in the memory and executable on the processor. When the visual speed measurement program is executed by the processor, the various steps of the visual speed measurement method described in the above embodiment are implemented.

[0186] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a visual speed measurement program, and when the visual speed measurement program is executed by a processor, the various steps of the visual speed measurement method described in the above embodiment are implemented.

[0187] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0188] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, system, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, system, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, system, article or device including the element.

[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment system can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a parking management device, an air conditioner, or a network device, etc.) to execute the system described in each embodiment of the present invention.

[0190] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A visual speed measurement method, characterized in that: The visual speed measurement method comprises: Determine the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture solution of the camera; Determining the compensation displacement of the camera according to a preset duration and the movement speed; Determining the detection displacement of the target to be detected in the video shot by the camera during the preset time length, including: determining the actual length of the target structure of the target to be detected in the video shot by the camera, and the pixel length of the target structure; determining a scale factor according to the actual length and the pixel length; determining the pixel distance moved by the target to be detected during the preset time length; determining the detection displacement corresponding to the target to be detected according to the scale factor and the pixel distance; The actual displacement is determined according to the compensation displacement and the detection displacement, and the actual speed of the target to be detected is determined according to the preset time length and the actual displacement.

2. The visual speed measurement method according to claim 1, characterized in that: Before the step of determining the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture calculation of the camera, the method further includes: Get the linear acceleration and angular acceleration of the inertial measurement unit; Determine a linear velocity and an angular velocity of the camera according to the linear acceleration and the angular acceleration; The attitude parameters of the camera are determined according to the linear velocity and the angular velocity, and the attitude solution result includes the attitude parameters.

3. The visual speed measurement method according to claim 2, characterized in that: The step of determining the linear velocity and angular velocity of the camera according to the linear acceleration and the angular acceleration comprises: Performing an integration operation on the linear acceleration within a preset time period to obtain the linear velocity; The angular velocity is obtained by performing an integration operation on the angular acceleration within a preset time period.

4. The visual speed measurement method according to claim 2, characterized in that: The step of determining the posture parameter of the camera according to the linear velocity and the angular velocity comprises: Determine the gravitational acceleration of the camera according to the angular velocity; Determine a gravity error value according to the normalized linear acceleration and the gravity acceleration; Determine a corrected gravity acceleration according to a preset proportionality coefficient, the gravity acceleration and the gravity error value; The attitude parameter is determined according to the angular velocity and the corrected gravitational acceleration.

5. The visual speed measurement method according to claim 4, characterized in that: The step of determining the gravity error value according to the normalized linear acceleration and the gravity acceleration comprises: Cross-multiply the normalized linear acceleration and the gravitational acceleration to calculate a target error; An integration operation is performed according to the target error within a preset time period to obtain the gravity error value.

6. The visual speed measurement method according to claim 2, characterized in that: The step of determining the movement speed of the camera based on the linear acceleration of the camera measured by the inertial measurement unit and the result of the posture calculation of the camera comprises: Determining a target matrix of the posture parameters; determining the gravitational acceleration of the camera according to the angular velocity; The movement speed of the camera is determined according to the target matrix, the gravitational acceleration and the linear acceleration.

7. A visual speed measuring device, characterized in that: The visual speed measuring device comprises: A determination module, configured to determine a movement speed of the camera based on a linear acceleration of the camera measured by an inertial measurement unit and a result of solving a posture of the camera; A first calculation module, used to determine the compensation displacement of the camera according to a preset duration and the movement speed; The second calculation module is used to determine the detection displacement of the target to be detected in the video shot by the camera during the preset time length, which includes: determining the actual length of the target structure of the target to be detected in the video shot by the camera, and the pixel length of the target structure; determining a scale factor according to the actual length and the pixel length; determining the pixel distance moved by the target to be detected within the preset time length; and determining the detection displacement corresponding to the target to be detected according to the scale factor and the pixel distance; The speed measurement module is used to determine the actual displacement according to the compensation displacement and the detection displacement, and to determine the actual speed of the target to be detected according to the preset time and the actual displacement.

8. A visual speed measurement device, characterized in that: The visual speed measurement device includes a memory, a processor, and a visual speed measurement program stored in the memory and executable on the processor. When the visual speed measurement program is executed by the processor, the various steps of the visual speed measurement method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a visual speed measurement program, and when the visual speed measurement program is executed by a processor, each step of the visual speed measurement method according to any one of claims 1 to 6 is implemented.

Images