A method for real-time measurement of dynamic camera motion parameters based on one-dimensional image information
By acquiring the current and previous frame images, randomly extracting one-dimensional image vectors, and utilizing median statistics, the problems of difficulty in determining reference objects and high computational resource consumption in camera motion parameter measurement are solved, enabling fast and accurate measurement of camera displacement and scaling parameters.
Patent Information
- Application Number
- CN202111609507.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-27
AI Technical Summary
When the camera is in motion, the measurement of motion parameters has problems such as difficulty in determining reference objects, high computational resources and time consumption, difficulty in feature recognition due to simple backgrounds, lack of consideration for rotation and scaling issues, and significant impact from image blurring and environmental changes.
By acquiring the current frame and the previous frame image, a one-dimensional image vector is randomly extracted, and the displacement parameter and scaling factor are calculated. The accuracy is improved by using the median statistical method. The one-dimensional image vector is used as a sample to reduce computing resources and improve speed and stability.
It enables rapid and accurate measurement of camera displacement parameters and scaling factors, reduces computational resource consumption, and improves the accuracy and stability of measurement results.
Smart Images

Figure CN114463434B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information. Background Art
[0002] In recent years, target recognition and tracking, 3D reconstruction, etc. have made rapid progress in the field of computer vision and have been widely used in the industrial field. Before further tracking, it is often necessary to perform preliminary preprocessing on the captured images, such as image denoising and camera distortion correction. For image processing when the camera is moving, since there is often motion blur and background transformation during the movement, if the camera's movement and scaling parameters are known, there will be better processing effects. However, in reality, such parameters are often difficult to obtain, so researchers often need to estimate the approximate movement and scaling parameters based on the recorded content. The accuracy of the estimated parameters will also limit the subsequent operational performance.
[0003] Traditional methods for estimating camera motion parameters mainly focus on image block recognition, feature extraction, etc. Similar methods require processing the entire image, which takes up relatively large computing resources and takes a long time to process. Since this process is only a preprocessing for subsequent operations, it requires fewer resources and the time taken by this process should be relatively small compared to the entire process. For images with simple backgrounds, the effects of traditional feature recognition and image block recognition methods will also be poor because the entire process cannot extract effective features. Parameter estimation for mobile cameras is a relatively subjective process because it is often necessary to specify a reference object. The displacement parameters and scaling factors of the camera need to be determined based on the selected reference object. However, in different application scenarios, there is often a lack of a reference object that can reach a consensus.
[0004] At present, there are mainly the following problems in measuring the motion parameters of a moving camera:
[0005] (1) It is difficult to determine the reference object for judging the camera movement parameters. Since the distance between the object in the field of view and the camera is different during the shooting process, and the lens itself has a certain degree of distortion during the shooting process, the movement parameters measured in different areas during the movement process are not the same;
[0006] (2) With the development of technology, the image resolution is getting higher and higher, and the time and computing resources consumed by the method based on positioning and image block prediction of camera movement distance are increasing;
[0007] (3) The image background may be very simple, and the background difference between two adjacent frames may be very small, which may make it impossible to identify specific movement parameters;
[0008] (4) During the movement process, the camera may not only move in translation, but may also rotate and zoom. At the same time, there may be changes in the environment such as lighting during the recording process, and the images captured during the movement may have motion blur problems. Summary of the invention
[0009] In view of the problems existing in the measurement of the movement parameters of a camera in a moving state, the present invention provides a method for real-time measurement of the movement parameters of a dynamic camera based on one-dimensional image information, wherein the movement parameters include a camera displacement parameter and a camera zoom coefficient, and the method comprises the following steps:
[0010] Step 1: Obtain the current frame image and the adjacent previous frame image captured by the dynamic camera to be measured;
[0011] Step 2: randomly extracting a number of one-dimensional image vectors within the threshold region of the previous frame image;
[0012] Step 3: Obtain the camera displacement parameters corresponding to the previous frame image, determine the initial position corresponding to each one-dimensional image vector after being moved according to the camera displacement parameters in the current frame image, and search for the one-dimensional vector that best matches the one-dimensional image vector in a neighborhood near the initial position, calculate the distance between the current position of the one-dimensional vector and the corresponding initial position, and obtain the movement acceleration of the current frame image;
[0013] Step 4: Calculate the displacement parameter of each one-dimensional vector according to the camera movement distance of the previous frame image and the movement acceleration of the current frame image corresponding to each one-dimensional vector;
[0014] Step 5: Take the median of the displacement parameters of all the one-dimensional vectors to obtain the camera displacement parameter corresponding to the current frame image.
[0015] Compared with the prior art, the present invention has the following beneficial effects:
[0016] (1) The method for real-time measurement of dynamic camera motion parameters based on one-dimensional image information proposed in the present invention selects a one-dimensional image vector as a sample, which can greatly save computing resources and has a faster computing speed. In addition, by using a statistical method of sampling a large number of samples and taking the median, the error of the final calculation can be reduced, thereby improving the accuracy of the measurement result;
[0017] (2) This method uses the first-order information of the displacement parameters, i.e., the change in the displacement parameters during the image movement process, to assist in determining the specific displacement parameters of the current frame image. This method not only has a fast calculation speed, but is also more stable than searching for matching vectors over a large range, thereby improving the stability of the entire process. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0019] Figure 1 It is a flow chart of the method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to the present invention;
[0020] Figure 2 is an example image in the present invention, wherein Figure 2 (a) is the previous frame image, Figure 2 (b) is the current frame image;
[0021] Figure 3 Schematic diagram of the camera displacement parameter calculation process in the present invention. DETAILED DESCRIPTION
[0022] The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and preferred embodiments.
[0023] like Figure 1 As shown, a method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information proposed by the present invention comprises the following steps:
[0024] 1. Input image
[0025] Step 1: Obtain a current frame image captured by the dynamic camera to be measured and a previous frame image adjacent to the current frame image, wherein the current frame image may be the second frame and subsequent images in the image sequence captured by the dynamic camera to be measured.
[0026] 2. Sampling process
[0027] The first is the selection of the threshold area. Since the recorded scene is constantly moving during the shooting process, if the sampling position happens to be at the boundary, the theoretical matching position of the next frame sample may be outside the image, resulting in the inability to find the matching position. Therefore, it is necessary to ensure that the sampling position is in the area where the scenes of two adjacent frames overlap, so it is generally necessary to pre-set a safety distance. The part of the image that is greater than the pre-set safety distance from each boundary of the image is the threshold area, such as Figure 2As shown in (a), the image area in the black frame is the threshold area. The value of the pre-set safety distance is determined by the maximum value of the maximum horizontal distance and the maximum vertical distance of any two adjacent frames of images captured by the dynamic camera to be measured. This method can ensure that all samples obtained in this area are matched in the next frame of the image. For example, for an image with a resolution of 1920×1080, if the horizontal and vertical distances of two adjacent frames of the image do not exceed 90 pixels, the safety distance can be selected as 90 pixels. In this way, the threshold area is the image area with a resolution of (1920-2×90)×(1080-2×90)=1740×900 from the center of the image to the outside.
[0028] The second is sample selection. This method uses a one-dimensional image vector as a sample. A one-dimensional image vector, different from a two-dimensional image vector (i.e., an image block), is information of a single row (column) in an image. Since the one-dimensional image vector in the present invention is extracted from the previous frame of image, the one-dimensional image vector actually refers to a single row of image information or a single column of image information in the previous frame of image, such as Figure 2 As shown in (a), the white line is the selected one-dimensional image vector. At the same time, it should be pointed out that the one-dimensional image vector extracted in the previous frame of the image can be all single-row image information or all single-column image information, or part of single-row image information or single-column image information. Color images are usually stored in the form of multiple channels in the computer, for example, RGB images are usually stored in the form of three channels. For an RGB image of size 224×224, it is usually represented in the computer as a 224×224×3 tensor. The one-dimensional image vector in this method does not consider the number of image channels, and the definition of one dimension is only proposed for the concept of the two-dimensional plane of the image. For example, in an RGB image, a one-dimensional image vector with a length of 40 is actually selected, and a tensor of size 40×3 is actually selected (3 is the number of channels of the image). Compared with selecting a two-dimensional image vector, although the one-dimensional image vector as a sample contains less information, it can greatly improve the calculation speed.
[0029] Step 2: Randomly extract several one-dimensional image vectors within the threshold region of the previous frame image.
[0030] This step is based on the above-mentioned sampling process, and randomly extracts a certain number of one-dimensional image vectors within the threshold area of the previous frame image. For example, a total of 100 to 500 one-dimensional image vectors are extracted. The more one-dimensional image vectors there are, the better the accuracy of the measurement results of the dynamic camera movement parameters. Sampling within the threshold area can ensure that the lines selected in the current frame image will not exceed the image boundary. By selecting a large number of one-dimensional image vectors, it can be ensured that most of its lines fall in the background area rather than the foreground area, so that the selected reference objects are unified as the background part of the image. Regarding the selected one-dimensional image vectors, it should be ensured that the standard deviation of their vector values is greater than the preset threshold to prevent the accuracy of the final measurement result from being reduced due to the one-dimensional image vector being located in an overly simple background area.
[0031] 3. Camera moving distance calculation process
[0032] like Figure 3 As shown, step three: obtain the camera displacement parameter corresponding to the previous frame image, determine the initial position corresponding to each one-dimensional image vector after moving according to the camera displacement parameter in the current frame image, and find the one-dimensional vector that best matches the one-dimensional image vector in the neighborhood near the initial position, calculate the distance between the current position of the one-dimensional vector and the corresponding initial position, and obtain the movement acceleration of the current frame image. The camera displacement parameter in the present invention is specifically expressed as a vector composed of horizontal displacement and vertical displacement, and the unit is pixel. For example, the camera displacement parameter measured for a certain frame image is (m, n), where "m" represents horizontal displacement, the size is |m|, and the positive and negative signs of m represent the same or opposite to the specified horizontal positive direction; "n" represents vertical displacement, the size is |n|, and the positive and negative signs of n represent the same or opposite to the specified vertical positive direction. The camera displacement parameter (m, n) means that the camera of this frame image is displaced by m pixels in the horizontal direction and n pixels in the vertical direction compared with the previous frame image.
[0033] In this step, the camera displacement parameters corresponding to the previous frame image are first obtained. When the previous frame image is the first frame image in the image sequence captured by the dynamic camera, the camera displacement parameters corresponding to the previous frame image are set to (0, 0). Since the displacement parameters of the previous frame image are known, it can be assumed that the current frame image has the same displacement parameters. At this time, the corresponding position of each one-dimensional image vector in the current frame image after moving according to the displacement parameters, that is, the initial position, is found, and then the one-dimensional vector that best matches the one-dimensional image vector is found in a smaller neighborhood near the initial position, such as Figure 2As shown in (b), the white line is the most matching one-dimensional vector corresponding to the one-dimensional image vector extracted from the previous frame. The distance between the current position of the most matching one-dimensional vector and its corresponding initial position is the change in the moving distance, that is, the moving acceleration of the current frame image (the displacement parameter measured in this method is the displacement of the camera corresponding to a certain frame image relative to the previous frame image, that is, the displacement change of each frame, in pixels / frame. According to the definition of physics, the measured result should be the speed of camera movement, so the change in moving distance calculated by this method should actually be the acceleration of camera movement. Since this method focuses on the displacement of two adjacent frames of images, the term "speed" is not used in the expression). For example, the position coordinates of one of the one-dimensional image vectors selected in the previous frame are (15, 20). It is known that the camera displacement parameters corresponding to the previous frame are (+20, +30), and the corresponding camera horizontal displacement and vertical displacement are +20 and +30 respectively. Then the matching vector of the one-dimensional image vector in the current frame should be found in a smaller neighborhood around the initial position (35, 50). Assuming that the position of the best matching one-dimensional vector relative to the initial position (35, 50) is (-3, -2), then the displacement parameters of the camera corresponding to the current frame should be (+20, +30) + (-3, -2) = (+17, +28), and the corresponding displacement parameter changes are +17 and +28.
[0034] Furthermore, the method for finding the one-dimensional vector that best matches the one-dimensional image vector in the neighborhood near the initial position is: respectively calculating the Manhattan distance between the one-dimensional image vector and all target vectors in the neighborhood near the initial position, that is, the one-norm of the vector difference. The smaller the one-norm of the vector difference, the higher the matching degree between the two. Therefore, the target vector corresponding to the minimum Manhattan distance is used as the one-dimensional vector that best matches the one-dimensional image vector. Due to the continuity of the moving speed, the search area of the vector during the execution process is also relatively small, and the search speed and accuracy will be much faster than a large-scale search.
[0035] Step 4: Based on the camera displacement parameter of the previous frame image and the calculated movement acceleration of each one-dimensional vector corresponding to the current frame image, the displacement parameter of each one-dimensional image vector can be calculated.
[0036] Step 5: By calculating the displacement parameters of all selected one-dimensional image vectors, take the median of the displacement parameters of all one-dimensional image vectors, that is, take the median of the horizontal displacement and the median of the vertical displacement in the displacement parameters of all one-dimensional image vectors respectively, and use the median of all horizontal displacements as the horizontal displacement in the camera displacement parameters, and use the median of all vertical displacements as the vertical displacement in the camera displacement parameters, and finally obtain the camera displacement parameters corresponding to the current frame image. By taking the median of the displacement parameters of all one-dimensional image vectors as the final calculation result, the error can be reduced through statistical methods to obtain a more accurate measurement result.
[0037] 4. Camera zoom factor calculation process
[0038] Step 6: Calculate the scaling factor corresponding to any one-dimensional image vector based on the position coordinates of the one-dimensional image vector in the previous frame, the moving distance of the one-dimensional image vector, the camera moving distance obtained by taking the median, and the center position coordinates of the previous frame.
[0039] According to the camera principle, the zoom process is relative to the center of the image. In the process of calculating the camera displacement parameters, the camera displacement can be well estimated. Therefore, this step can directly estimate the image zoom factor by selecting the one-dimensional image vector at the possible position of the current frame image. For the i-th one-dimensional image vector, the formula for the corresponding zoom factor calculated by the vector is:
[0040]
[0041] Among them, s i is the position coordinate of the one-dimensional image vector in the previous frame of image, d i is the displacement parameter of the one-dimensional image vector, s mid is the coordinate of the center position of the previous frame image, d med It is the camera displacement parameter corresponding to the current frame image finally calculated, that is, the median of the displacement parameters calculated by all one-dimensional image vectors.
[0042] Step 7: Take the median of the scaling coefficients corresponding to all one-dimensional image vectors to obtain the camera scaling coefficient, and finally output the movement parameters of the dynamic camera.
[0043] The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information proposed by the present invention selects a one-dimensional image vector as a sample, which can greatly save computing resources and has a faster computing speed. Moreover, by sampling a large number of samples and taking the median statistical method, the error of the final calculation can be reduced, thereby improving the accuracy of the measurement result. If the displacement parameter calculated by a certain frame is greatly different from the actual value, the actual displacement parameter will gradually approach the actual displacement parameter in the subsequent frames through the method of the present invention. At the same time, the method uses the first-order information of the displacement parameter, that is, the change amount of the displacement parameter during the image movement process, to assist in determining the specific displacement parameter of the current frame image. Not only is the calculation speed fast, but it is also more stable than a large-scale search for matching vectors, thereby improving the stability of the entire process.
[0044] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0045] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information, wherein the movement parameters include camera displacement parameters and camera zoom coefficients, characterized in that: The following steps are involved: Step 1: Obtain the current frame image and the adjacent previous frame image captured by the dynamic camera to be measured; Step 2: randomly extracting a number of one-dimensional image vectors within the threshold region of the previous frame image; Step 3: Obtain the camera displacement parameters corresponding to the previous frame image, determine the initial position corresponding to each one-dimensional image vector after being moved according to the camera displacement parameters in the current frame image, and search for the one-dimensional vector that best matches the one-dimensional image vector in a neighborhood near the initial position, calculate the distance between the current position of the one-dimensional vector and the corresponding initial position, and obtain the movement acceleration of the current frame image; Step 4: Calculate the displacement parameter of each one-dimensional image vector according to the camera displacement parameter of the previous frame image and the movement acceleration of the current frame image corresponding to each one-dimensional vector; Step 5: taking the median of the displacement parameters of all the one-dimensional image vectors to obtain the camera displacement parameter corresponding to the current frame image; Step 6: Calculate the scaling factor corresponding to any one of the one-dimensional image vectors according to the position coordinates of the one-dimensional image vector in the previous frame image, the displacement parameter of the one-dimensional image vector, the camera displacement parameter obtained by taking the median, and the center position coordinates of the previous frame image; Step 7: Take the median of the scaling coefficients corresponding to all the one-dimensional image vectors to obtain the camera scaling coefficient.
2. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: The formula for calculating the scaling factor corresponding to the one-dimensional image vector in step 6 is: Among them, s i is the position coordinate of the one-dimensional image vector in the previous frame image, d i is the displacement parameter of the one-dimensional image vector, s mid is the center position coordinate of the previous frame image, d med is the camera displacement parameter obtained by taking the median.
3. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: The threshold area of the previous frame image is an image portion in the previous frame image that is farther away from each boundary than a preset safety distance.
4. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 3, characterized in that: The preset safety distance is determined by the maximum value of the maximum horizontal distance and the maximum vertical distance of any two adjacent frames of images captured by the dynamic camera to be measured.
5. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: The method for finding the one-dimensional vector that best matches the one-dimensional image vector in a neighborhood near the initial position is: calculating the Manhattan distances between the one-dimensional image vector and all target vectors in the neighborhood near the initial position respectively, and taking the target vector corresponding to the minimum Manhattan distance as the one-dimensional vector that best matches the one-dimensional image vector.
6. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: When the previous frame image is the first frame image in the image sequence captured by the dynamic camera, the camera displacement parameter corresponding to the previous frame image is zero.
7. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: The standard deviation of the vector value of the one-dimensional image vector extracted in step 2 is greater than a preset threshold.
8. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: The one-dimensional image vector is a single row of image information or a single column of image information in the previous frame of image.
9. The method for real-time measurement of dynamic camera movement parameters based on one-dimensional image information according to claim 1, characterized in that: The total number of the one-dimensional image vectors extracted in step 2 is 100 to 500.
