Target positioning method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of camera measurement, and discloses a target positioning method and device, computer equipment and a storage medium, and the method comprises the steps: determining a model point and candidate feature points corresponding to a target point on a current frame image; determining a reference feature point according to the distance between the model point and the candidate feature point; determining matching point pairs according to the reference feature points; constructing a similar transformation matrix according to the matching point pairs, and determining a registration error corresponding to the similar transformation matrix; when the registration error is smaller than a preset threshold value, determining the similar transformation matrix as a target transformation matrix; determining a search area on the next frame of image through the covariance matrix; fitting a plurality of prediction points in the search area by using a least square method to obtain a fitting function, and solving an extreme point of the fitting function; and correcting the extreme point according to the target transformation matrix to obtain the position and speed of the target point on the next frame of image. Based on the technical scheme of the invention, the precision and efficiency of target point positioning can be improved.

Description

Technical Field

[0001] The present invention relates to the field of camera measurement technology, and in particular to a target positioning method, device, computer equipment and storage medium. Background Art

[0002] In the actual monitoring application of visual measurement systems, in order to save costs and expand the monitoring range, it is usually necessary to set or select multiple features as test points at different depths of field in the field of view of each camera. For example, when measuring the deformation of the steel structure relative to the main structure after installation, multiple luminous signs or reflective signs are usually arranged at key parts of the steel structure as test points. For visual measurement systems, the measurement platform will be in a state of shaking or rapid vibration due to factors such as geological conditions and construction. In order to meet the measurement accuracy requirements, visual measurement systems usually use small field of view lenses. Under this condition, even if the measurement platform is deflected at a small angle, it will cause drastic changes in the image coordinates of the test point in the field of view, resulting in inaccurate positioning results.

[0003] Therefore, how to ensure the accuracy of target positioning when the measurement platform is deflected becomes a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0004] In view of this, the present invention provides a target positioning method, device, computer equipment and storage medium to solve the problem in the related art that even a small angle deflection of the measuring platform will cause drastic changes in the image coordinates of the measured point in the field of view, resulting in inaccurate positioning results.

[0005] In a first aspect, the present invention provides a target positioning method, the target positioning method comprising:

[0006] Determine the candidate feature points corresponding to the model points and the target points on the current frame image;

[0007] Determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point; determine a matching point pair according to the reference feature point;

[0008] Constructing a similarity transformation matrix according to the matching point pairs, and determining a registration error corresponding to the similarity transformation matrix;

[0009] When the registration error is less than a preset threshold, determining the similarity transformation matrix as a target transformation matrix;

[0010] Determine a search area for a target point on a next frame of image by using a covariance matrix of a Kalman filter; the search area includes a plurality of prediction points;

[0011] Using the least square method to fit the plurality of predicted points in the search area to obtain a fitting function, and solving the extreme value point of the fitting function to achieve sub-pixel level positioning of the target point;

[0012] The extreme point is corrected according to the target transformation matrix to obtain the position and speed of the target point on the next frame image.

[0013] In an optional implementation, when the number of the candidate feature points is less than or equal to a preset number, determining the reference feature points among the candidate feature points according to the distances between the model points and the candidate feature points includes:

[0014] Determine, according to the distance between the model point and the candidate feature point, a candidate feature point whose target distance ratio is less than or equal to a distance threshold among the candidate feature points; the target distance ratio is the ratio of the closest distance to the second closest distance among the first two candidate feature points closest to the model point;

[0015] The candidate feature points whose target distance ratios are less than or equal to the distance threshold are determined as the reference feature points.

[0016] In an optional implementation, when the number of the candidate feature points is greater than a preset number, determining the matching point pairs according to the reference feature points includes:

[0017] Determine the candidate feature points in the middle area according to the distribution of the candidate feature points;

[0018] Determine the reference feature point based on the candidate feature points in the middle area;

[0019] The reference feature points determined based on the candidate feature points in the middle area are combined in pairs to obtain the matching point pairs.

[0020] In an optional implementation, determining the registration error corresponding to the similarity transformation matrix includes:

[0021] Determine the projection feature point corresponding to the model point according to the similarity transformation matrix, and determine the neighborhood feature point corresponding to the projection feature point; the neighborhood feature point is a candidate feature point within the neighborhood range of the projection feature point, and the neighborhood range is a spatial range with the projection feature point as the center and a preset distance as the radius;

[0022] A registration error corresponding to the similarity transformation matrix is ​​determined according to the projection feature points and the neighborhood feature points.

[0023] In an optional implementation, determining the registration error corresponding to the similarity transformation matrix according to the projection feature points and the neighborhood feature points includes:

[0024] Determining the similarity between the projected feature point and the neighborhood feature point;

[0025] The registration error corresponding to the similarity transformation matrix is ​​determined according to the similarity between the projection feature point and the neighborhood feature point.

[0026] In an optional implementation, the target positioning method further includes:

[0027] When the registration error is greater than or equal to the preset threshold, the preset threshold or the distance threshold is updated according to the number of the neighborhood feature points, and the following steps are re-executed until the registration error is less than the preset threshold:

[0028] Determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point, determine a matching point pair according to the reference feature point, construct a similarity transformation matrix according to the matching point pair, and determine a registration error corresponding to the similarity transformation matrix.

[0029] In an optional implementation, fitting the plurality of prediction points in the search area using the least squares method to obtain a fitting function, and solving the extreme value point of the fitting function includes:

[0030] In the search area, a plurality of the prediction points are fitted using the least square method to obtain a two-dimensional Gaussian function, and the extreme value points of the two-dimensional Gaussian function are solved.

[0031] In a second aspect, the present invention provides a target positioning device, the target positioning device comprising:

[0032] A first processing module is used to determine candidate feature points corresponding to the model point and the target point on the current frame image;

[0033] A second processing module is used to determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point; and determine a matching point pair according to the reference feature point;

[0034] A third processing module is used to construct a similarity transformation matrix according to the matching point pairs, and determine a registration error corresponding to the similarity transformation matrix;

[0035] A fourth processing module, configured to determine the similarity transformation matrix as a target transformation matrix when the registration error is less than a preset threshold;

[0036] A fifth processing module, used to determine a search area for a target point on a next frame of image by using a covariance matrix of a Kalman filter; the search area includes a plurality of prediction points;

[0037] A sixth processing module, configured to fit the plurality of predicted points in the search area using the least square method to obtain a fitting function, and solve the extreme value point of the fitting function to achieve sub-pixel level positioning of the target point;

[0038] The seventh processing module is used to correct the extreme point according to the target transformation matrix to obtain the position and speed of the target point on the next frame image.

[0039] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the target positioning method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the target positioning method of the first aspect or any corresponding embodiment thereof.

[0041] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the target positioning method of the first aspect or any corresponding embodiment thereof.

[0042] The technical solution provided by the present invention has the following technical effects:

[0043] Based on the technical solution of the present invention, by determining the candidate feature points corresponding to the model points and the target points on the current frame image, and determining the reference feature points among the candidate feature points according to the distance between the model points and the candidate feature points, the influence of noise and mismatch on target positioning can be reduced, and the accuracy and robustness of target positioning can be improved. The search area of ​​the target point on the next frame image is determined by the covariance matrix of the Kalman filter, which can reduce the scope of the search area and improve the real-time performance of target positioning. By fitting multiple prediction points in the search area using the least squares method to obtain a fitting function and solving the extreme points of the fitting function, the sub-pixel level positioning of the target point can be achieved, and the accuracy of target positioning can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1is a schematic diagram of a flow chart of a target positioning method according to an embodiment of the present invention;

[0046] Figure 2 is a structural block diagram of a target positioning device according to an embodiment of the present invention;

[0047] Figure 3 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0049] In the actual monitoring application of visual measurement systems, in order to save costs and expand the monitoring range, it is usually necessary to set or select multiple features as test points at different depths of field in the field of view of each camera. For example, when measuring the deformation of the steel structure relative to the main structure after installation, multiple luminous signs or reflective signs are usually arranged at key parts of the steel structure as test points. For visual measurement systems, the measurement platform will be in a state of shaking or rapid vibration due to factors such as geological conditions and construction. In order to meet the measurement accuracy requirements, visual measurement systems usually use small field of view lenses. Under this condition, even if the measurement platform is deflected at a small angle, it will cause drastic changes in the image coordinates of the test point in the field of view, resulting in inaccurate positioning results. Under outdoor conditions, this change is usually difficult to model or filter and track the features, which brings great challenges to fast and high-precision feature matching.

[0050] Generally speaking, compared with the matching method based on feature points, the grayscale-based sub-pixel matching algorithm has higher matching accuracy. However, in order to solve the problem of camera shaking, the traditional multi-feature tracking and positioning method based on template matching needs to set a search window. If the search window is set too small, it is easy to cause features to go out of the search window and cause missed matches. If the search window is set too large, it is easy to cause multiple similar features to appear in the same search window and cause mismatches. Both of the above situations will cause feature tracking failure.

[0051] Even if the template matching method based on the adaptive window is used, it is difficult to select a suitable window when the measurement platform shakes rapidly, which may cause a large error when a certain image jumps.

[0052] Therefore, the embodiments of the present invention provide a target positioning method, apparatus, computer equipment and storage medium to solve the problem that rapid shaking of the measurement platform in a field measurement environment easily causes feature positioning and matching failures, so as to solve the above problems.

[0053] According to an embodiment of the present invention, an embodiment of a target positioning method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0054] Figure 1 4 is a flow chart of a target positioning method according to an embodiment of the present invention.

[0055] like Figure 1 As shown, a target positioning method provided in an embodiment of the present invention is applied to the case where a measuring platform shakes. The target positioning method includes:

[0056] S101: Determine candidate feature points corresponding to the model points and the target points on the current frame image.

[0057] In this embodiment, after completing the calibration of the camera's internal and external parameters, the spatial point is projected onto the corresponding camera imaging surface, and the image point coordinates and sequence corresponding to the spatial point can be obtained. The image point in this state is used as the model point. The model point can be represented in the form of a set or sequence, for example, it can be represented as: M = {p i , ..., p i , ..., p m Where m represents the number of model points, and at least one model point should be determined. Feature detection can be performed on the target point on the current frame image to determine the candidate feature points corresponding to the target point. Specifically, luminous signs and reflective signs can be fixedly installed at the position of the point to be tested. A narrow-band filter can be set according to the wavelength of the light source. Without setting a short exposure, the target point often appears as a bright spot on the image, and the background area is black. By setting a suitable threshold, the candidate position of the target point can be quickly segmented to obtain the candidate feature point. The candidate feature points can be represented in the form of a set or sequence, for example, it can be represented as: N = {q 1 , ..., q j , ..., q n}, where n represents the number of candidate feature points, and at least two candidate feature points should be determined.

[0058] S102: Determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point, and determine a matching point pair according to the reference feature point.

[0059] As an example, when the number of candidate feature points is less than or equal to a preset number, the above S102 determines the reference feature points among the candidate feature points according to the distance between the model point and the candidate feature point, specifically including:

[0060] According to the distance between the model point and the candidate feature point, the candidate feature point whose target distance ratio is less than or equal to the distance threshold is determined among the candidate feature points. The candidate feature point whose target distance ratio is less than or equal to the distance threshold is determined as the reference feature point. The target distance ratio is the ratio of the closest distance to the second closest distance of the first two candidate feature points closest to the model point.

[0061] In this embodiment, the preset number and the distance threshold can be designed and modified according to actual needs. As an example, based on test experience, the preset number is usually 2, and the distance threshold is usually [0.8, 1.2].

[0062] For example, for model point A, the feature point closest to model point A is candidate feature point B, and the feature point second closest to model point A is candidate feature point C. The target distance ratio is the ratio of the closest distance (the distance between model point A and candidate feature point B) to the second closest distance (the distance between model point A and candidate feature point C). When the target distance ratio is less than or equal to the distance threshold, candidate feature point B and candidate feature point C are determined as the reference feature points of model point A. The reference feature points of model point A (candidate feature point B and candidate feature point C) can be determined as a set of matching point pairs.

[0063] When the number of candidate feature points is less than or equal to the preset number, according to the above technical solution, for each model point, the candidate feature points whose target distance ratio is less than or equal to the distance threshold are determined according to the distance between the single model point and the candidate feature point. The candidate feature points whose target distance ratio is less than or equal to the distance threshold are determined as the reference feature points corresponding to the single model point, and the reference feature points corresponding to each model point are obtained.

[0064] In this embodiment, the reference feature corresponding to a single model point may be determined as a single set of matching point pairs.

[0065] In this field, there are many models for describing image transformation, and commonly used models include perspective, affine, similarity, and rigid body transformation. For a fixed measuring station formed by a parallel camera network, the shake of the measuring system is relatively small, where the rotation is usually in the degree level, and the translation is usually in the centimeter level. For a telephoto lens, the scaling of the horizontal and vertical axes can be considered to be consistent. Therefore, the transformation between the candidate feature points detected in the current frame image and the model points can be modeled as a similarity transformation model, and the image transformation is specifically achieved through a similarity transformation matrix.

[0066] In this embodiment, a similarity transformation matrix between the model point and the candidate feature point can be predefined, and the transformation relationship between any point p and q in the model point is as follows:

[0067] q = Ψ(p, A) = Ap

[0068] Where p=(p x p y 1) represents the homogeneous coordinates of the model point, q = (q x q y 1) T represents the homogeneous coordinates of the candidate feature point, Ψ(p, A) represents the transformation relationship between any point p and q in the model point, represents the similarity transformation matrix, λ represents the scaling factor, represents the rotation factor, θ represents the rotation angle, and t represents the translation factor.

[0069] According to the transformation relationship between any point p and q in the model point, each group of p and q can generate 2 constraints, and the degree of freedom of the similarity transformation matrix A is 4, so at least 2 groups of matching point pairs are required to solve the similarity transformation matrix A. In order to ensure the high efficiency of solving the similarity transformation matrix, the technical solution of the present invention does not use an exhaustive method to blindly match model points and candidate feature points at any position as matching point pairs, but makes full use of the slight shaking of the measurement platform, and the priori that the measured point in the field of view has a small movement or the reference point has no movement, and selects candidate feature points whose target distance ratio is less than or equal to the distance threshold, that is, the reference feature points, and determines the matching point pairs based on the reference feature points. Based on the above technical solution, the number of similarity transformation matrices A generated is reduced, and it can ensure that at least one hypothetical transformation is close to the real transformation.

[0070] As another example, when the number of candidate feature points is greater than a preset number, for example, when the number of candidate feature points is greater than 2, determining matching point pairs according to the reference feature points in S102 specifically includes:

[0071] The candidate feature points in the middle area are determined according to the distribution of the candidate feature points.

[0072] The reference feature points are determined based on the candidate feature points in the middle area. The candidate feature points in the middle area can be determined as the reference feature points.

[0073] The reference feature points determined based on the candidate feature points in the middle area are combined in pairs to obtain matching point pairs. The candidate feature points (reference feature points) in the middle area can be combined in pairs to obtain matching point pairs.

[0074] In this embodiment, when the number of candidate feature points is greater than the preset number, since the features in the middle area of ​​the field of view are more likely to appear in the camera's field of view when the camera shakes, the candidate feature points in the middle area can be selected as reference feature points, and the reference feature points determined based on the candidate feature points in the middle area can be combined in pairs to obtain matching point pairs.

[0075] In this embodiment, the reference feature points determined based on the candidate feature points in the middle area are combined in pairs to obtain matching point pairs to solve the similarity transformation matrix, thereby further improving the generation efficiency of the hypothetical transformation.

[0076] In this embodiment, considering that there are a large number of candidate feature points in the middle area, more matching point pairs will be generated, resulting in the generation of more similarity transformation matrices, which will affect the calculation efficiency. Therefore, before the benchmark feature points determined according to the candidate feature points in the middle area are combined in pairs to obtain matching point pairs, it is also possible to determine whether the number of candidate feature points in the middle area is greater than the number threshold. If the number of candidate feature points in the middle area is greater than the number threshold, the middle area can be re-divided to reduce the area of ​​the middle area and the number of candidate feature points in the middle area, which not only ensures the calculation accuracy but also improves the calculation efficiency.

[0077] S103: constructing a similarity transformation matrix according to the matching point pairs, and determining a registration error corresponding to the similarity transformation matrix.

[0078] In this embodiment, a single set of matching point pairs corresponds to a similarity transformation matrix. A single set of matching point pairs includes two reference feature points.

[0079] In this embodiment, the transformation methods applied include but are not limited to similarity transformation. Similarity transformation between features can be achieved through at least two feature points, affine transformation between features can be achieved through at least three feature points, and homography transformation between features can be achieved through at least four feature points.

[0080] S104: When the registration error is less than a preset threshold, the similarity transformation matrix is ​​determined as a target transformation matrix.

[0081] In this embodiment, when the registration error is greater than or equal to a preset threshold, the preset threshold or distance threshold is updated according to the number of neighborhood feature points, and the following technical scheme S102-S103 is re-executed until the registration error is less than the preset threshold: determine the benchmark feature points among the candidate feature points according to the distance between the model point and the candidate feature point, determine the matching point pairs according to the benchmark feature points, construct a similarity transformation matrix according to the matching point pairs, and determine the registration error corresponding to the similarity transformation matrix.

[0082] In this embodiment, iterative training can be performed based on the Random Sample Consensus (RANSAC) algorithm to determine the registration error corresponding to the similarity transformation matrix, and when the registration error is greater than or equal to a preset threshold, it is determined to perform the next iterative training, and when the registration error is less than the preset threshold, the similarity transformation matrix obtained by the current iterative training is determined as the target transformation matrix. Considering that the model points and the candidate feature points are not initially corresponding, and the corresponding relationship between the model points and the candidate feature points is unknown, in this embodiment, the RANSAC algorithm can be used to establish the corresponding relationship under the RANSAC iteration framework.

[0083] S105: Determine a search area for target points on the next frame of image using the covariance matrix of the Kalman filter.

[0084] In this embodiment, the search area includes a plurality of prediction points. As an example, the Kalman filter used in this embodiment may be a correlation Kalman filter.

[0085] S106: Fitting a plurality of prediction points in the search area using the least square method to obtain a fitting function, and solving the extreme value point of the fitting function to achieve sub-pixel level positioning of the target point.

[0086] In this embodiment, as an example, the fitting function includes but is not limited to a two-dimensional Gaussian function, and a two-variable quadratic polynomial or other surface function may also be used. When the fitting function is a two-dimensional Gaussian function, the above S106 specifically includes: fitting multiple prediction points in the search area using the least squares method to obtain a two-dimensional Gaussian function, and solving the extreme value point of the two-dimensional Gaussian function.

[0087] In this embodiment, a fitting function may be obtained by fitting a plurality of prediction points in the search region using the least square method, and the extreme value point of the fitting function may be solved.

[0088] S107: Correct the extreme point according to the target transformation matrix to obtain the position and speed of the target point on the next frame image.

[0089] In an optional implementation, determining the registration error corresponding to the similarity transformation matrix in S103 specifically includes:

[0090] Sa1: Determine the projection feature points corresponding to the model points according to the similarity transformation matrix, and determine the neighborhood feature points corresponding to the projection feature points.

[0091] In this embodiment, the neighborhood feature point is a candidate feature point within the neighborhood range of the projection feature point, and the neighborhood range is a spatial range with the projection feature point as the center and a preset distance as the radius.

[0092] In this embodiment, after obtaining the similarity transformation matrix A, the model point M = {p 1 , ..., p i , ..., p m} is transformed to the new image position to obtain the projected feature points, recorded as In order to select the correct hypothetical transformation, that is, the target transformation matrix, it is necessary to test the randomly generated similarity transformation matrix, so the projection feature points It is aligned with the candidate feature point N, and the validity of the assumed transformation is determined by calculating the alignment error. To avoid searching the entire image, due to the limited range of the camera's own motion, the feature point only moves in a local range on the image plane, and it is necessary to search in the neighborhood of the projected feature point. The corresponding neighborhood feature points can be stored in the form of a set or sequence, for example:

[0093]

[0094] Among them, N(q i ) represents the set of neighborhood feature points, σ(·) is the sequence number of the neighborhood feature points within the neighborhood, n nbr is the number of neighborhood feature points within the neighborhood, take n nbr ≥m, Represents a single neighborhood feature point.

[0095] Sa2: Determine the registration error corresponding to the similarity transformation matrix based on the projection feature points and neighborhood feature points.

[0096] In an optional implementation, Sa2 determines the registration error corresponding to the similarity transformation matrix according to the projection feature points and the neighborhood feature points, specifically including:

[0097] Determine the similarity between the projected feature points and the neighborhood feature points.

[0098] The registration error corresponding to the similarity transformation matrix is ​​determined according to the similarity between the projected feature points and the neighborhood feature points.

[0099] In this embodiment, the similarity between the projected feature point and the neighborhood feature point can be represented by a distance, and the distance includes but is not limited to the Euclidean distance. As an example, the similarity between the projected feature point and the neighborhood feature point can be represented by the Euclidean distance. When the similarity between the projected feature point and the neighborhood feature point is represented by the Euclidean distance, the registration error corresponding to the similarity transformation matrix can be determined by the following formula.

[0100]

[0101] Among them, S(M, N, A) represents the configuration error corresponding to the similarity transformation matrix A, Represents the projected feature points and neighborhood feature point q j The similarity between (Euclidean distance), represents p determined according to the similarity transformation matrix A i The corresponding projected feature points, when the feature points are blocked, use S max Controls excessive penalties for correct transformations, Represents the projected feature points Neighborhood feature points within the neighborhood range.

[0102] In this embodiment, a hypothesis-test model based on the RANSAC algorithm can be constructed, and the similarity transformation matrix constructed based on the matching point pairs can be used as the hypothesis transformation. Through iterative training, the mismatched point pairs can be eliminated to improve the matching accuracy. In the hypothesis-test model, the number of neighborhood feature points within the neighborhood range is used as the size of the inner point set. During the iterative training process, the inner point probability or the number of iterations of the RANSAC algorithm can be updated according to the size of the inner point set, and the following S102-S103 technical solutions can be re-executed until the registration error is less than the preset threshold value. The similarity transformation matrix obtained after the cycle ends is the target transformation matrix.

[0103] In the present invention, camera shaking will produce slight posture changes, and the position and speed changes of the target point caused by the shaking of the camera are holistic and global. By modeling the holistic changes caused by the shaking of the camera, the rapid positioning of multiple features can be achieved, thereby updating the features that are out of the field of view, blocked, and re-entering the field of view. By constructing a hypothesis-testing model based on the RANSAC algorithm, a fast and robust estimation of the transformation parameters is achieved, and then the model points are projected to the new image position using similar transformation parameters, solving the problem of rapid and rough positioning of multiple features under camera shaking. In the present invention, the target transformation matrix obtained according to the technical solutions of S101-S104 above can be used to eliminate the positioning error caused by the shaking of the measurement platform or the camera itself.

[0104] In order to solve the problem of large image coordinate jumps at the target point due to camera shake, the present invention uses an associated Kalman filter, which predicts the position and speed of the target point in the next frame in each frame. The purpose of this is not only to predict the target position when the target is blocked and improve the anti-blocking ability, but also to determine the target's motion state in the event of lens field deflection. The target's motion state is reflected in the continuity of the target point position in the image. The associated Kalman filter predicts the position and speed of the target point, and the position and speed of the target point can be represented by a state vector. The state vector of the target point can be updated using the associated Kalman filter. and the covariance matrix State Vector and the covariance matrix The expressions are:

[0105] Where F represents the state transfer matrix, Represents the state vector of the target point on the previous frame image, P k-1 represents the covariance matrix corresponding to the previous frame image, and Q represents the process noise covariance matrix. Where (x, y) is the x and y coordinates of the target point on the current frame image, (v x , v y ) is the speed of the target point in the current frame image in the x and y directions. When the tracking lens jumps, the lens field of view deflects. At this time, it can be considered that the coordinates of the target point change while the speed of the target point remains unchanged. The associated Kalman filter will automatically correct the two-dimensional image coordinates of the target point. After correction by the target transformation matrix A, the state vector of the target point is updated as: The relationship between (x, y) and (x′, y′) is:

[0106] The associated Kalman filter can predict the position of the target point in real time by updating the state vector of the target point when the target point is occluded. It can also compensate for the displacement caused by the movement of the target point and the background through the target transformation matrix when the camera moves.

[0107] According to the Kalman filter, the position of the target point on the next frame of the image can be predicted. According to the covariance matrix, the search area of ​​the target point on the next frame of the image can be determined, and the template matching in the small feature area can obtain the pixel-level coarse positioning of the model point on the real-time image. In order to obtain the sub-pixel positioning accuracy, it is necessary to use the least squares method to fit the correlation coefficient matrix of each prediction point in the neighborhood of the coarse positioning pixel, and determine the position of the target point on the next frame of the image by solving the extreme point or the maximum point of the fitting function. The position obtained after correcting the extreme point or the maximum point through the target transformation matrix is ​​the position of the target point on the next frame of the image.

[0108] Usually, for features with Gaussian distribution such as light spots, it is suitable to be fitted with a two-dimensional Gaussian function, which can be expressed as:

[0109]

[0110] Where f(x, y) represents a two-dimensional Gaussian function, denoted by f(x, y), A represents the amplitude, σ x and σ y Represents the standard deviation in the x and y directions respectively, (x 0 ,y 0) represents the coordinates of the extreme points of the two-dimensional Gaussian fitting function, and (x, y) represents the coordinates of the prediction points.

[0111] Taking the logarithm of both sides of the two-dimensional Gaussian function to eliminate the exponential form, expanding and sorting it out, we can get:

[0112]

[0113] If a Gaussian surface is fitted to an n×n pixel neighborhood, then n 2 flnf, this n 2 The system of equations consisting of flnf is expressed by a matrix:

[0114] Wq=b

[0115] Where W represents the matrix of Geng 5, and the i-th row is expressed as:

[0116]

[0117] q represents a 5×1 vector composed of Gaussian parameters:

[0118]

[0119] b represents a column vector of size 1, and the elements of the i-th row are:

[0120] b i =f i lnf i

[0121] The least square method can be used to solve Wq=b to obtain the values ​​of each element of the vector x, and then decompose it to obtain the various unknown parameters of the Gaussian function:

[0122]

[0123] Among them, q 1 ,q 2 ,q 3 ,q 4 ,q 5 represents a constant, the extreme point of the two-dimensional Gaussian fitting function (x 0 ,y 0 ) is what we are looking for. Through the above steps, sub-pixel positioning of multiple prediction points can be achieved.

[0124] Aiming at the problem that rapid shaking of the measuring platform in a field measuring environment easily causes the failure of target point positioning and matching, the present invention proposes a multi-target tracking and sub-pixel target positioning method based on structural constraints and Kalman filtering, models the similarity transformation between the model point and the candidate feature point as a similarity transformation matrix, and constructs a registration framework based on the RANSAC algorithm to achieve pixel-level correspondence accuracy, and then uses the associated Kalman filter to quickly predict the position and speed of the target point after the jump. On this basis, the size of the search area can be adaptively determined according to the covariance matrix of the associated Kalman filter, and the pixel-level position of the target point is determined based on a small range template matching of the grayscale feature. Finally, the sub-pixel feature positioning accuracy can be obtained by constructing a two-dimensional Gaussian fitting function.

[0125] Traditional image target tracking methods based on Kalman filtering or association filtering cannot cope with sudden changes or large jumps in target center coordinates caused by rapid camera movement or jitter. The present invention can eliminate the influence of platform rigid motion by means of the overall similarity transformation of candidate feature points corresponding to the target point, and track and predict the new position and speed of the target point by means of association Kalman filtering, adaptively determine the search area related to template matching sub-pixels, and improve positioning efficiency. The present invention effectively solves the problem faced by the visual measurement system in the actual measurement process that the target point coordinates move rapidly or change significantly due to vibration or jitter caused by geological conditions and neighboring disturbances, which leads to target tracking failure. The technical solution based on the present invention can significantly improve the automation and intelligence level of the visual measurement system, significantly reduce human intervention, and improve measurement efficiency.

[0126] The technical solution of the present invention can reduce the influence of noise and mismatch on target positioning and improve the accuracy and robustness of target positioning by determining the candidate feature points corresponding to the model point and the target point on the current frame image, and determining the reference feature points among the candidate feature points according to the distance between the model point and the candidate feature point. By fitting a plurality of prediction points in the search area using the least squares method to obtain a fitting function and solving the extreme point of the fitting function, it is possible to achieve sub-pixel level positioning of the target point and improve the accuracy of target positioning. The search area of ​​the target point on the next frame image is determined by the covariance matrix of the Kalman filter, and the range of the search area can be adaptively updated to improve the real-time performance of target positioning.

[0127] It should be noted that the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

[0128] In this embodiment, a target positioning device is also provided, which is used to implement the above-mentioned embodiment and optional implementation methods, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceivable.

[0129] Figure 2 is a structural block diagram of a target positioning device according to an embodiment of the present invention.

[0130] This embodiment provides a target positioning device, such as Figure 2 As shown, the target positioning device includes:

[0131] The first processing module 11 is used to determine candidate feature points corresponding to the model points and the target points on the current frame image.

[0132] The second processing module 12 is used to determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point, and to determine a matching point pair according to the reference feature point.

[0133] The third processing module 13 is used to construct a similarity transformation matrix according to the matching point pairs, and determine the registration error corresponding to the similarity transformation matrix.

[0134] The fourth processing module 14 is used to determine the similarity transformation matrix as the target transformation matrix when the registration error is less than a preset threshold.

[0135] The fifth processing module 15 is used to determine a search area for target points on the next frame of image by using the covariance matrix of the Kalman filter. The search area includes a plurality of prediction points.

[0136] The sixth processing module 16 is used to fit multiple prediction points in the search area using the least square method to obtain a fitting function, and solve the extreme point of the fitting function to achieve sub-pixel level positioning of the target point.

[0137] The seventh processing module 17 is used to correct the extreme point according to the target transformation matrix to obtain the position and speed of the target point on the next frame image.

[0138] In an optional embodiment, the second processing module 12 includes a first processing unit.

[0139] The first processing unit of the second processing module 12 is used to determine, when the number of candidate feature points is less than or equal to a preset number, candidate feature points whose target distance ratio is less than or equal to a distance threshold among the candidate feature points according to the distance between the model point and the candidate feature point. The candidate feature points whose target distance ratio is less than or equal to the distance threshold among the candidate feature points are determined as reference feature points. The target distance ratio is the ratio of the closest distance to the second closest distance among the first two candidate feature points closest to the model point.

[0140] In an optional implementation, the second processing module 12 further includes a second processing unit.

[0141] The second processing unit of the second processing module 12 is used to determine the candidate feature points in the middle area according to the distribution of the candidate feature points when the number of the candidate feature points is greater than the preset number. Determine the reference feature points according to the candidate feature points in the middle area. Combine the reference feature points determined according to the candidate feature points in the middle area in pairs to obtain matching point pairs.

[0142] In an optional implementation, the third processing module 13 includes a first processing unit.

[0143] The first processing unit of the third processing module 13 includes:

[0144] The first processing subunit is used to determine the projection feature points corresponding to the model points according to the similarity transformation matrix, and determine the neighborhood feature points corresponding to the projection feature points. The neighborhood feature points are candidate feature points within the neighborhood range of the projection feature points, and the neighborhood range is a spatial range with the projection feature point as the center and a preset distance as the radius.

[0145] The second processing subunit is used to determine the registration error corresponding to the similarity transformation matrix according to the projection feature points and the neighborhood feature points.

[0146] In an optional implementation, the second processing subunit of the first processing unit under the third processing module 13 is specifically used to determine the similarity between the projected feature points and the neighborhood feature points. The registration error corresponding to the similarity transformation matrix is ​​determined according to the similarity between the projected feature points and the neighborhood feature points.

[0147] In an optional implementation, the target positioning device further includes: a judgment module.

[0148] The judgment module is used to update the preset threshold or distance threshold according to the number of neighborhood feature points when the registration error is greater than or equal to the preset threshold, and re-execute the following steps until the registration error is less than the preset threshold:

[0149] The reference feature points among the candidate feature points are determined according to the distance between the model point and the candidate feature points, the matching point pairs are determined according to the reference feature points, a similarity transformation matrix is ​​constructed according to the matching point pairs, and the registration error corresponding to the similarity transformation matrix is ​​determined.

[0150] In an optional implementation, the sixth processing module 16 is specifically configured to use the least square method to fit multiple prediction points in the search area to obtain a two-dimensional Gaussian function, and solve the extreme value points of the two-dimensional Gaussian function.

[0151] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0152] The target positioning device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0153] The embodiment of the present invention also provides a computer device having the above Figure 2 The target positioning device shown.

[0154] See also Figure 3 , Figure 3 Schematic diagram of the hardware structure of the computer device according to the embodiment of the present invention. Figure 3 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In an optional embodiment, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 10 is taken as an example.

[0155] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0156] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0157] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function. The data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In an optional embodiment, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] The memory 20 may include a volatile memory, such as a random access memory. The memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive. The memory 20 may also include a combination of the above-mentioned types of memory.

[0159] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0160] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0161] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0162] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A target positioning method, characterized in that: The target positioning method comprises: Determine the candidate feature points corresponding to the model points and the target points on the current frame image; Determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point; determine a matching point pair according to the reference feature point; Constructing a similarity transformation matrix according to the matching point pairs, and determining a registration error corresponding to the similarity transformation matrix; When the registration error is less than a preset threshold, determining the similarity transformation matrix as a target transformation matrix; Determine a search area for a target point on a next frame of image by using a covariance matrix of a Kalman filter; the search area includes a plurality of prediction points; Using the least square method to fit the plurality of predicted points in the search area to obtain a fitting function, and solving the extreme value point of the fitting function to achieve sub-pixel level positioning of the target point; The extreme point is corrected according to the target transformation matrix to obtain the position and speed of the target point on the next frame image.

2. The method according to claim 1, characterized in that When the number of the candidate feature points is less than or equal to a preset number, determining the reference feature points among the candidate feature points according to the distances between the model points and the candidate feature points includes: Determine, according to the distance between the model point and the candidate feature point, a candidate feature point whose target distance ratio is less than or equal to a distance threshold among the candidate feature points; the target distance ratio is the ratio of the closest distance to the second closest distance among the first two candidate feature points closest to the model point; The candidate feature points whose target distance ratios are less than or equal to the distance threshold are determined as the reference feature points.

3. The method according to claim 2, characterized in that When the number of the candidate feature points is greater than a preset number, determining the matching point pairs according to the reference feature points includes: Determine the candidate feature points in the middle area according to the distribution of the candidate feature points; Determine the reference feature point based on the candidate feature points in the middle area; The reference feature points determined based on the candidate feature points in the middle area are combined in pairs to obtain the matching point pairs.

4. The method according to claim 2, characterized in that: The determining the registration error corresponding to the similarity transformation matrix includes: Determine the projection feature point corresponding to the model point according to the similarity transformation matrix, and determine the neighborhood feature point corresponding to the projection feature point; the neighborhood feature point is a candidate feature point within the neighborhood range of the projection feature point, and the neighborhood range is a spatial range with the projection feature point as the center and a preset distance as the radius; A registration error corresponding to the similarity transformation matrix is ​​determined according to the projection feature points and the neighborhood feature points.

5. The method according to claim 4, characterized in that The determining, according to the projection feature points and the neighborhood feature points, a registration error corresponding to the similarity transformation matrix comprises: Determining the similarity between the projected feature point and the neighborhood feature point; The registration error corresponding to the similarity transformation matrix is ​​determined according to the similarity between the projection feature point and the neighborhood feature point.

6. The method according to claim 4, characterized in that The target positioning method further includes: When the registration error is greater than or equal to the preset threshold, the preset threshold or the distance threshold is updated according to the number of the neighborhood feature points, and the following steps are re-executed until the registration error is less than the preset threshold: Determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point, determine a matching point pair according to the reference feature point, construct a similarity transformation matrix according to the matching point pair, and determine a registration error corresponding to the similarity transformation matrix.

7. The method according to claim 1, characterized in that The step of fitting the plurality of prediction points in the search area using the least square method to obtain a fitting function and solving the extreme value point of the fitting function includes: In the search area, a plurality of the prediction points are fitted using the least square method to obtain a two-dimensional Gaussian function, and the extreme value points of the two-dimensional Gaussian function are solved.

8. A target positioning device, characterized in that: The target positioning device comprises: A first processing module is used to determine candidate feature points corresponding to the model point and the target point on the current frame image; A second processing module is used to determine a reference feature point among the candidate feature points according to the distance between the model point and the candidate feature point; and determine a matching point pair according to the reference feature point; A third processing module is used to construct a similarity transformation matrix according to the matching point pairs, and determine the registration error corresponding to the similarity transformation matrix; A fourth processing module, configured to determine the similarity transformation matrix as a target transformation matrix when the registration error is less than a preset threshold; A fifth processing module, used to determine a search area for a target point on a next frame of image by using a covariance matrix of a Kalman filter; the search area includes a plurality of prediction points; A sixth processing module, configured to fit the plurality of predicted points in the search area using the least square method to obtain a fitting function, and solve the extreme value point of the fitting function to achieve sub-pixel level positioning of the target point; The seventh processing module is used to correct the extreme point according to the target transformation matrix to obtain the position and speed of the target point on the next frame image.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the steps of the target positioning method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the steps of the target positioning method according to any one of claims 1 to 7.

Images