Image positioning method, device and equipment for planar object and medium

Abstract

The application relates to the technical field of image positioning, and discloses a planar object image positioning method, device, equipment and medium, wherein the method comprises the following steps: performing image mapping on a template image by using a homography matrix; calculating a first residual error vector of a first mapping image and a current target image; calculating a weight matrix and a Jacobian matrix of a parameter vector according to the first residual error vector; updating the parameter vector by using Tikhonov regularization according to the Jacobian matrix, the weight matrix and the first residual error vector, so as to obtain a first updated parameter vector; updating the homography matrix by using the first updated parameter vector, so as to obtain a first updated homography matrix; judging whether the first updated parameter vector is less than or equal to a parameter vector threshold value, and if yes, taking the first updated homography matrix as a final homography matrix; the method can reduce the influence of abnormal points with possible obstacles on image positioning, and can also prevent the drift phenomenon caused by the presence of obstacles.

Description

Technical Field

[0001] This application relates to the field of image positioning technology, and for example to an image positioning method, apparatus, device and medium for planar objects. Background Technology

[0002] Image localization is a crucial area in computer vision, forming the foundation of object tracking. Given a template image, the computer vision system continuously locates the template image within the target image. Efficient second-order minimization methods are classic image matching algorithms frequently used in object tracking applications. However, when the tracked target encounters obstacles, the success rate and accuracy of these methods drop drastically. Furthermore, the localization results of efficient second-order minimization methods drift as obstacles move.

[0003] In summary, existing efficient second-order minimization methods applied to object tracking suffer from problems such as a sharp drop in target tracking success rate and accuracy when the tracked target encounters obstacles, and drift in localization results as the obstacles move. Summary of the Invention

[0004] The purpose of this application is to provide an image localization method, apparatus, device, and medium for planar objects, which can solve the problems of existing efficient second-order minimization methods in object tracking, where the target tracking success rate and accuracy drop sharply when the tracked target encounters obstacles, and the localization results drift as the obstacle moves. To achieve the above objective, this application provides an image localization method for planar objects, including:

[0005] Obtain the template image, the homography matrix, and the parameter vector corresponding to the homography matrix;

[0006] The template image is mapped using the homography matrix to obtain a first mapped image;

[0007] Calculate the first residual vector between the first mapped image and the current target image;

[0008] Calculate the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector;

[0009] The parameter vector is updated using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain the first updated parameter vector.

[0010] The homography matrix is ​​updated using the first update parameter vector to obtain the first updated homography matrix;

[0011] Determine whether the first updated parameter vector is less than or equal to the parameter vector threshold. If so, use the first updated homography matrix as the final homography matrix. The final homography matrix is ​​used to locate the template image.

[0012] This application provides an image positioning device for a planar object, comprising:

[0013] The parameter acquisition module is used to acquire the template image, the homography matrix, and the parameter vector corresponding to the homography matrix;

[0014] The first image mapping module is used to perform image mapping on the template image using the homography matrix to obtain a first mapped image;

[0015] The first residual vector calculation module is used to calculate the first residual vector between the first mapped image and the current target image;

[0016] The first matrix calculation module is used to calculate the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector.

[0017] The first parameter vector update module is used to update the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix and the first residual vector to obtain the first updated parameter vector.

[0018] The first homography matrix update module is used to update the homography matrix using the first update parameter vector to obtain the first updated homography matrix;

[0019] The first determination module is used to determine whether the first updated parameter vector is less than or equal to the parameter vector threshold. If so, the first updated homography matrix is ​​used as the final homography matrix. The final homography matrix is ​​used to locate the template image.

[0020] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image positioning method for a planar object as described in any of the above claims and / or the steps of the image positioning method for a planar object as described in any of the above claims.

[0021] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image positioning method for a planar object as described in any of the preceding claims and / or the steps of the image positioning method for a planar object as described in any of the preceding claims.

[0022] This application discloses an image localization method for planar objects, comprising: acquiring a template image, a homography matrix, and a parameter vector corresponding to the homography matrix; mapping the template image using the homography matrix to obtain a first mapped image; calculating a first residual vector between the first mapped image and the current target image; calculating a Jacobian matrix of a weight matrix and a parameter vector based on the first residual vector; updating the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain a first updated parameter vector; updating the homography matrix using the first updated parameter vector to obtain a first updated homography matrix; determining whether the first updated parameter vector is less than or equal to a parameter vector threshold; if so, using the first updated homography matrix as the final homography matrix; and using the final homography matrix to localize the template image. When information about an obstacle appears at the target location, the localization result will drift as the obstacle moves. Using Tikhonov regularization to update the parameter vector can obtain a more stable first updated parameter vector, that is, the new localization position is more biased towards the original position, thereby preventing the drift phenomenon caused by the appearance of obstacles. The weight matrix is ​​calculated based on the first residual vector. Each term of the first residual vector includes a pixel difference. A weight coefficient is assigned to each pixel difference. The weight coefficients form a weight matrix. The larger the pixel difference, the greater the possibility that there is an obstacle at the position of the corresponding pixel. The smaller the weight coefficient corresponding to the pixel difference, the less the impact of abnormal points that may contain obstacles on image localization, thereby improving the success rate and accuracy of target tracking. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating an image localization method for a planar object according to one embodiment.

[0024] Figure 2 This is a flowchart illustrating the calculation of the Jacobian matrix of the weight matrix and parameter vectors in one embodiment.

[0025] Figure 3 This is a flowchart illustrating the normalized template image and the current target image in one embodiment.

[0026] Figure 4 This is a schematic diagram illustrating the process of image mapping of a template image according to one embodiment;

[0027] Figure 5 This is a schematic diagram of an image mapping embodiment;

[0028] Figure 6 A schematic block diagram of an image positioning device for a planar object according to an embodiment;

[0029] Figure 7 This is a schematic block diagram of the structure of a computer device according to one embodiment.

[0030] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0033] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0034] In one embodiment, refer to Figure 1 This is a flowchart illustrating the image positioning method for planar objects disclosed in this application, including:

[0035] S1: Obtain the template image, the homography matrix, and the parameter vector corresponding to the homography matrix.

[0036] During image localization, the user provides a template image, and the computer vision system continuously locates the position of the template image within the current target image.

[0037] The template image is a two-dimensional image, and the homography matrix can be used as a representation of the localization result of the template image. That is, the position of the template image can be located in the current target image through the homography matrix.

[0038] The parameter vector of the homography matrix is ​​a = [a0, a1, a2, a3, a4, a5, a6, a7]. T , where T represents the matrix transpose.

[0039] The homography matrix can be represented by a parameter vector as follows:

[0040]

[0041] Among them, a i Let A be the i-th parameter in the parameter vector of the homography matrix. i Let represent the basis of the vector space corresponding to the i-th parameter in the parameter vector, e denotes the exponential function, and H(a) is the homography matrix.

[0042] S2: Use the homography matrix to perform image mapping on the template image to obtain a first mapped image.

[0043] Multiply the homography matrix by the pixel coordinates of each template image to obtain the mapped pixel coordinates;

[0044] The mapped pixel value is determined based on the mapped pixel coordinates;

[0045] The first mapped image is determined based on the mapped pixel coordinates and the mapped pixel values.

[0046] The first mapped image can reflect the accuracy of the homography matrix. The higher the similarity between the first mapped image and the template image, the higher the accuracy of the homography matrix; the lower the similarity between the first mapped image and the template image, the lower the accuracy of the homography matrix.

[0047] S3: Calculate the first residual vector between the first mapped image and the current target image.

[0048] The first mapped image and the current target image have the same size, and the total number of pixels in both images is q. The first mapped image and the current target image are converted into row vectors. The difference between the row vector corresponding to the first mapped image and the row vector corresponding to the current target image is taken to obtain the first residual vector.

[0049] The formula for calculating the first residual vector is as follows:

[0050]

[0051]

[0052] in, Let x represent the pixel value of the i-th pixel in the first mapped image. i I(x) represents the i-th pixel in the template image. i() represents the pixel value of the i-th pixel in the template image. This represents the i-th residual value, which is the difference between two pixel values. Let represent the first residual vector, which is the transpose of the vector consisting of all residual values.

[0053] S4: Calculate the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector.

[0054] Calculate the first-order partial derivative of the first residual vector with respect to the basis coefficients of the vector space to obtain the first intermediate result;

[0055] Based on the first intermediate result, the Jacobian matrix of the parameter vector is calculated according to the following formula:

[0056]

[0057] in, Let be the Jacobian matrix of the parameter vector. J(0) is the first intermediate result, and J(0) is the second intermediate result.

[0058] The difference between the first residual vector and the median of the first residual vector is used as the standardized first residual vector;

[0059] Calculate the absolute median difference of the standardized first residual vector;

[0060] Calculate the standard deviation of the standardized first residual vector based on the absolute median;

[0061] The weight matrix is ​​calculated based on the absolute median difference and the standard deviation of the standardized first residual vector.

[0062] The weight matrix is ​​calculated based on the first residual vector. Each item of the first residual vector includes a pixel difference. A weight coefficient is assigned to each pixel difference. The weight coefficients form a weight matrix. The larger the pixel difference, the greater the possibility that there is an obstacle at the position of the corresponding pixel. The smaller the weight coefficient corresponding to the pixel difference, the less the impact of abnormal points that may contain obstacles on image localization.

[0063] S5: Update the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain the first updated parameter vector.

[0064] The first updated parameter vector is calculated according to the following formula:

[0065]

[0066] in, For the first updated parameter vector, Let be the Jacobian matrix, W be the weight matrix, T be the matrix transpose operation, I be the identity matrix, λ be the regularization parameter, and y(0) be the first residual vector.

[0067] Preferably, the regularization parameter is set to 500000.

[0068] Using Tikhonov regularization to update the parameter vector results in a more stable first update parameter vector, meaning the new positioning position is more biased towards the original position, thus preventing drift caused by obstacles.

[0069] S6: Update the homography matrix using the first update parameter vector to obtain the first updated homography matrix.

[0070] Calculate the first incremental homography matrix corresponding to the first updated parameter vector;

[0071] The product of the first incremental homography matrix and the homography matrix is ​​used as the first updated homography matrix.

[0072] If the homography matrix does not satisfy the condition that the first residual vector is 0, the first incremental homography matrix needs to be calculated based on the homography matrix. The first incremental homography matrix is ​​an incremental mapping.

[0073] The first updated homography matrix reflects the mapping relationship from the template image to an image region in the current target image. The product of the first incremental homography matrix and the homography matrix is ​​used as the first updated homography matrix, making the first residual vector closer to 0.

[0074] S7: Determine whether the first updated parameter vector is less than or equal to the parameter vector threshold. If so, use the first updated homography matrix as the final homography matrix. The final homography matrix is ​​used to locate the template image.

[0075] Determine whether the L2 norm of the first updated parameter vector is less than or equal to the parameter vector threshold. If so, use the first updated homography matrix as the final homography matrix.

[0076] The final homography matrix reflects the mapping relationship between the template image and the current target image. The region in the current target image that corresponds to the template image can be located through the final homography matrix.

[0077] An image localization method for a planar object according to an embodiment of this application includes obtaining a template image, a homography matrix, and a parameter vector corresponding to the homography matrix; mapping the template image using the homography matrix to obtain a first mapped image; calculating a first residual vector between the first mapped image and the current target image; calculating a Jacobian matrix of a weight matrix and a parameter vector based on the first residual vector; updating the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain a first updated parameter vector; updating the homography matrix using the first updated parameter vector to obtain a first updated homography matrix; determining whether the first updated parameter vector is less than or equal to a parameter vector threshold; if so, using the first updated homography matrix as the final homography matrix; and using the final homography matrix to localize the template image. When information about an obstacle appears at the target location, the localization result will drift as the obstacle moves. Using Tikhonov regularization to update the parameter vector can obtain a more stable first updated parameter vector, that is, the new localization position is more biased towards the original position, thereby preventing the drift phenomenon caused by the appearance of obstacles. The weight matrix is ​​calculated based on the first residual vector. Each term of the first residual vector includes a pixel difference. A weight coefficient is assigned to each pixel difference. The weight coefficients form a weight matrix. The larger the pixel difference, the greater the possibility that there is an obstacle at the position of the corresponding pixel. The smaller the weight coefficient corresponding to the pixel difference, the less the impact of abnormal points that may contain obstacles on image localization, thereby improving the success rate and accuracy of target tracking.

[0078] In one embodiment, refer to Figure 2 The step of calculating the Jacobian matrix of the weight matrix and the parameter vector based on the first residual vector includes:

[0079] S41: Calculate the first-order partial derivative of the first residual vector with respect to the basis coefficients of the vector space to obtain the first intermediate result.

[0080] The formula for calculating the first intermediate result is as follows:

[0081]

[0082] in, This is the first intermediate result. Let be the first residual vector. It is the first-order partial derivative of the first residual vector with respect to the basis coefficients of the vector space.

[0083] S42: Calculate the Jacobian matrix of the parameter vector based on the first intermediate result.

[0084] Based on the first intermediate result, the Jacobian matrix of the parameter vector is calculated according to the following formula:

[0085]

[0086] in, Let be the Jacobian matrix of the parameter vector. J(0) is the first intermediate result, and J(0) is the second intermediate result.

[0087] S43: The difference between the first residual vector and the median of the first residual vector is used as the standardized first residual vector.

[0088] The formula for calculating the standardized first residual vector is as follows:

[0089]

[0090] in, Let y(0) be the first residual vector and median(y(0)) be the median of the first residual vector.

[0091] S44: Calculate the absolute median difference of the standardized first residual vector.

[0092] The formula for calculating the absolute median is as follows:

[0093]

[0094] Where MAD represents the absolute median difference, and median represents the median operation. This represents the absolute value of the standardized first residual vector.

[0095] S45: Calculate the standard deviation of the standardized first residual vector based on the absolute median difference.

[0096] The formula for calculating the standard deviation of the standardized first residual vector is as follows:

[0097] σ = 1.4826 * MAD;

[0098] Where σ is the standard deviation of the standardized first residual vector.

[0099] S46: Calculate the weight matrix based on the absolute median difference and the standard deviation of the standardized first residual vector.

[0100] The formula for calculating the weighting coefficient is as follows:

[0101]

[0102] k = c * σ;

[0103] Among them, w i Let be the i-th weight coefficient, and k be the weight coefficient threshold. Let be the i-th standardized first residual vector, and c be the harmonic variable.

[0104] Preferably, the harmonic variable is set to 3.27957.

[0105] Generate a diagonal matrix, and fill the first to the last weight coefficients into the diagonal of the diagonal matrix in sequence to obtain the weight matrix, which has a size of q×q.

[0106] The weight matrix is ​​calculated based on the first residual vector. Each item of the first residual vector includes a pixel difference. A weight coefficient is assigned to each pixel difference. The weight coefficients form a weight matrix. The larger the pixel difference, the greater the possibility that there is an obstacle at the position of the corresponding pixel. The smaller the weight coefficient corresponding to the pixel difference, the less the impact of abnormal points that may contain obstacles on image localization.

[0107] As described above, calculating the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector includes calculating the first partial derivative of the first residual vector with respect to the basis coefficients of the vector space, obtaining a first intermediate result. The Jacobian matrix of the parameter vector is calculated based on the first intermediate result, and the difference between the first residual vector and its median is used as the standardized first residual vector. The absolute median difference of the standardized first residual vector is calculated, and the standard deviation of the standardized first residual vector is calculated based on the absolute median difference. The weight matrix is ​​then calculated based on the absolute median difference and the standard deviation of the standardized first residual vector. Each term of the first residual vector includes a pixel difference, and a weight coefficient is assigned to each pixel difference. These weight coefficients form the weight matrix. A larger pixel difference indicates a higher probability of an obstacle at the corresponding pixel location, while a smaller weight coefficient reduces the impact of outliers that may contain obstacles on image localization.

[0108] In one embodiment, refer to Figure 3 Before calculating the Jacobian matrix of the weight matrix and the parameter vector based on the first residual vector, the method further includes:

[0109] S31': Calculate the image gradient of the template image.

[0110] The template image includes gradients in the x-direction and gradients in the y-direction. Calculate the gradient matrix of the template image in the x-direction and the gradient matrix in the y-direction.

[0111] S32': Filter the template image pixels according to the image gradient to obtain a pixel index set.

[0112] The template image pixels are filtered using the following formula:

[0113]

[0114] Wherein, G is the set of pixel indices, and i is the pixel index of the template image. The square of the gradient of the i-th pixel of the template image in the x-direction. q is the square of the gradient of the i-th pixel of the template image in the y-direction, g is the gradient threshold, and q is the total number of pixels in the template image.

[0115] By setting a gradient threshold g, template image pixels can be filtered. The larger the gradient threshold g, the fewer template image pixels are filtered; the smaller the gradient threshold g, the more template image pixels are filtered.

[0116] If the gradient threshold g is too small, the image localization method of this application embodiment will fail to converge; if the gradient threshold g is too large, the total number of pixels in the template image will be too large, and the time to update the parameter vector using Tikhonov regularization will be long, resulting in a long computation time for the image localization method of this application embodiment.

[0117] By filtering the pixels of the template image, computation time can be reduced and computation efficiency improved.

[0118] Preferably, the gradient threshold g is set to 12.

[0119] S33': Normalize the template image and the current target image according to the number of pixel indices in the pixel index set.

[0120] In practical applications, factors such as ambient light can cause significant differences in the brightness and contrast of the tracked target in the current target image compared to the given template image, thereby affecting the positioning effect of the image positioning method in this embodiment.

[0121] The formula for normalizing the template image is as follows:

[0122] I(x i )=I(x i )-∑ i∈G I(x i ) / q′;

[0123] Where, x i Let I(x) be the i-th pixel in the template image. i Let q' be the pixel value of the i-th template image pixel, q' be the total number of pixel indices in the pixel index set, and i∈G indicate that the i-th template image pixel belongs to the pixel index set.

[0124] The formula for normalizing the current target image is as follows:

[0125]

[0126] in, This represents the i-th pixel in the current target image. This represents the pixel value of the i-th pixel in the current target image.

[0127] By normalizing the template image and the current location image, the problem of inconsistent brightness between the template image and the current location image can be solved.

[0128] As described above, before calculating the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector, the process includes calculating the image gradient of the template image, filtering the pixels of the template image based on the image gradient to obtain a pixel index set, and normalizing the template image and the current target image based on the number of pixel indices in the pixel index set. Filtering the template image pixels reduces computation time and improves computational efficiency. Normalizing the template image and the current localization image resolves the issue of inconsistent brightness between them.

[0129] In one embodiment, after determining whether the first updated parameter vector is less than or equal to the parameter vector threshold, the method further includes:

[0130] S81: If the first updated parameter vector is greater than the parameter vector threshold, then the current target image is normalized according to the first updated homography matrix.

[0131] Step S81 is similar to step S33', except that the homography matrix in step S33' is replaced with the first updated homography matrix, which will not be described in detail here.

[0132] S82: Use the first updated homography matrix to perform image mapping on the template image to obtain a second mapped image.

[0133] Step S82 is similar to step S2, except that the homography matrix in step S2 is replaced with the first updated homography matrix, which will not be described in detail here.

[0134] S83: Calculate the second residual vector between the second mapped image and the current target image.

[0135] Step S83 is similar to step S3, except that the first mapping image in step S3 is replaced with the second mapping image, which will not be described in detail here.

[0136] S84: Calculate the Jacobian matrix of the weight matrix and the parameter vector based on the second residual vector.

[0137] Step S84 is similar to step S4, except that the first residual vector in step S4 is replaced with the second residual vector, which will not be described in detail here.

[0138] S85: Update the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the second residual vector to obtain the second updated parameter vector.

[0139] Step S85 is similar to step S5, except that the first residual vector in step S5 is replaced with the second residual vector, which will not be elaborated here. It is worth noting that the Jacobian matrix in step S85 has been updated in terms of specific values ​​compared to the Jacobian matrix in step S5, and the weight matrix in step S85 has also been updated in terms of specific values ​​compared to the weight matrix in step S5.

[0140] S86: Update the homography matrix using the second update parameter vector to obtain the second updated homography matrix.

[0141] Step S86 is similar to step S6, except that the first update parameter vector in step S6 is replaced with the second update parameter vector, which will not be described in detail here.

[0142] S87: Determine whether the second updated parameter vector is less than or equal to the parameter vector threshold. If so, use the second updated homography matrix as the final homography matrix.

[0143] Step S87 is similar to step S7, except that the first update parameter vector in step S7 is replaced with the second update parameter vector, which will not be described in detail here.

[0144] If the second updated parameter vector is greater than the parameter vector threshold, it means that the accuracy of mapping the template image to the current target image through the second updated homography matrix is ​​not high. It is necessary to continue to update the second updated homography matrix and repeat steps S81 to S87 until the Nth updated parameter vector is less than or equal to the parameter vector threshold. Then, the Nth updated homography matrix is ​​used as the final homography matrix.

[0145] As described above, the homography matrix can be continuously updated through multiple iterations, thereby improving the positioning accuracy of the homography matrix. When the updated parameter vector is less than or equal to the parameter vector threshold, the iteration is terminated, resulting in a final homography matrix with high positioning accuracy.

[0146] In one embodiment, refer to Figures 4-5 The step of using the homography matrix to perform image mapping on the template image to obtain a first mapped image includes:

[0147] S21: Multiply the homography matrix and the pixel coordinates of each template image of the template image to obtain the mapped pixel coordinates.

[0148] The homography matrix is ​​represented as The i-th pixel of the template image is represented as x. i , These are the mapped pixel coordinates.

[0149] Figure 5 In this context, x represents a template image pixel in the template image, x′ represents a mapped image pixel in the first mapped image, the first mapped image is an image region in the current target image, and H represents a homography matrix.

[0150] S22: Determine the mapped pixel value based on the mapped pixel coordinates.

[0151] The mapped pixel coordinates obtained through the homography matrix may be decimals or integers. The mapped pixel coordinates are rounded to obtain the integer form of the mapped pixel coordinates.

[0152] S23: Determine the first mapped image based on the mapped pixel coordinates and the mapped pixel value.

[0153] The pixel value corresponding to the mapped pixel coordinates in the current target image is used as the mapped pixel value, and the first mapped image is determined based on the mapped pixel coordinates and the corresponding mapped pixel value.

[0154] The first mapped image and the template image have the same size and the same total number of pixels.

[0155] As described above, image mapping of the template image using a homography matrix to obtain a first mapped image includes multiplying the homography matrix and the coordinates of each template image pixel to obtain mapped pixel coordinates. Mapped pixel values ​​are determined based on these coordinates, and the first mapped image is determined based on both the coordinates and the pixel values. The first mapped image has the same size as the template image, and the total number of pixels in both images is the same.

[0156] In one embodiment, after using the first updated homography matrix as the final homography matrix, the method further includes:

[0157] The template image is mapped using the final homography matrix to obtain the final mapped image. The template image is the image of a planar object in the world coordinate system, and the final mapped image is the image of the planar object in the image coordinate system.

[0158] The template image can be viewed as a two-dimensional plane. The homography matrix can map points on the two-dimensional plane to two-dimensional points in the image, that is, a two-dimensional point-to-point mapping.

[0159] The final mapped image is an image region in the current target image. The final mapped image may be obtained from the template image through rotation and / or scaling transformation.

[0160] The more accurate the final homography matrix, the more accurate the final mapped image obtained by using the final homography matrix for image mapping, that is, the more accurate the region of the template image is located in the current target image.

[0161] As described above, after using the first updated homography matrix as the final homography matrix, the process further includes using the final homography matrix to perform image mapping on the template image to obtain the final mapped image. The template image is the image of the planar object in the world coordinate system, and the final mapped image is the image of the planar object in the image coordinate system. The more accurate the final homography matrix, the more accurate the final mapped image obtained by using the final homography matrix for image mapping, that is, the more accurately the region of the template image is located in the current target image.

[0162] In one embodiment, updating the homography matrix using the first update parameter vector to obtain a first updated homography matrix includes:

[0163] S61: Calculate the first incremental homography matrix corresponding to the first updated parameter vector.

[0164] Replace the parameter vector in the homography matrix expression in step S1 with the first updated parameter vector, and replace the basis of the vector space corresponding to the i-th parameter in the parameter vector with the basis of the vector space corresponding to the i-th parameter in the first updated parameter vector to obtain the first incremental homography matrix.

[0165] S62: The product of the first incremental homography matrix and the homography matrix is ​​used as the first updated homography matrix.

[0166] The homography matrix is ​​adjusted by the first incremental homography matrix. The first updated homography matrix can more accurately locate the template image in the current target image compared to the homography matrix.

[0167] As described above, updating the homography matrix using the first update parameter vector to obtain the first updated homography matrix includes calculating the first incremental homography matrix corresponding to the first update parameter vector, and using the product of the first incremental homography matrix and the homography matrix as the first updated homography matrix. By adjusting the homography matrix using the first incremental homography matrix, the first updated homography matrix can more accurately locate the template image in the current target image compared to the homography matrix.

[0168] Reference Figure 6 This is a schematic block diagram of an image positioning device for a planar object disclosed in this application. The device includes:

[0169] The parameter acquisition module 10 is used to acquire the template image, the homography matrix, and the parameter vector corresponding to the homography matrix;

[0170] The first image mapping module 20 is used to perform image mapping on the template image using the homography matrix to obtain a first mapped image;

[0171] The first residual vector calculation module 30 is used to calculate the first residual vector between the first mapped image and the current target image;

[0172] The first matrix calculation module 40 is used to calculate the Jacobian matrix of the weight matrix and the parameter vector based on the first residual vector;

[0173] The first parameter vector update module 50 is used to update the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix and the first residual vector to obtain the first updated parameter vector.

[0174] The first homography matrix update module 60 is used to update the homography matrix using the first update parameter vector to obtain a first updated homography matrix;

[0175] The first determination module 70 is used to determine whether the first updated parameter vector is less than or equal to the parameter vector threshold. If so, the first updated homography matrix is ​​used as the final homography matrix. The final homography matrix is ​​used to locate the template image.

[0176] In one embodiment, the first matrix calculation module 40 further includes:

[0177] The first intermediate result calculation unit is used to calculate the first-order partial derivative of the first residual vector with respect to the basis coefficients of the vector space to obtain the first intermediate result.

[0178] The Jacobian matrix calculation unit is used to calculate the Jacobian matrix of the parameter vector based on the first intermediate result according to the following formula:

[0179]

[0180] in, Let be the Jacobian matrix of the parameter vector. J(0) is the first intermediate result, and J(0) is the second intermediate result.

[0181] The standardized first residual vector calculation unit is used to take the difference between the first residual vector and the median of the first residual vector as the standardized first residual vector;

[0182] An absolute median calculation unit is used to calculate the absolute median of the standardized first residual vector;

[0183] A standard deviation calculation unit is used to calculate the standard deviation of the standardized first residual vector based on the absolute median.

[0184] The weight matrix calculation unit is used to calculate the weight matrix based on the absolute median difference and the standard deviation of the standardized first residual vector.

[0185] In one embodiment, the image positioning device for the planar object further includes:

[0186] An image gradient calculation module is used to calculate the image gradient of the template image;

[0187] The template image pixel filtering module is used to filter the template image pixels according to the image gradient to obtain a set of pixel indices;

[0188] The first normalization module is used to normalize the template image and the current target image according to the number of pixel indices in the pixel index set.

[0189] In one embodiment, the image positioning device for the planar object further includes:

[0190] The second normalization module is used to normalize the current target image according to the first update homography matrix if the first update parameter vector is greater than the parameter vector threshold.

[0191] The second image mapping module uses the first updated homography matrix to perform image mapping on the template image to obtain a second mapped image;

[0192] The second residual vector calculation module is used to calculate the second residual vector between the second mapped image and the current target image;

[0193] The second matrix calculation module is used to calculate the Jacobian matrix of the weight matrix and the parameter vector based on the second residual vector;

[0194] The second parameter vector update module is used to update the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix and the second residual vector to obtain the second updated parameter vector.

[0195] The second homography matrix update module is used to update the homography matrix using the second update parameter vector to obtain a second updated homography matrix;

[0196] The second determination module is used to determine whether the second updated parameter vector is less than or equal to the parameter vector threshold. If so, the second updated homography matrix is ​​used as the final homography matrix.

[0197] In one embodiment, the first image mapping module 20 further includes:

[0198] The mapped pixel coordinate calculation unit is used to multiply the homography matrix and each template image pixel coordinate of the template image to obtain the mapped pixel coordinates;

[0199] A mapped pixel value determination unit is used to determine the mapped pixel value based on the mapped pixel coordinates;

[0200] The first mapped image determination unit is used to determine the first mapped image based on the mapped pixel coordinates and the mapped pixel values.

[0201] In one embodiment, the first parameter vector update module 50 further includes:

[0202] The first parameter vector update unit is used to calculate the first updated parameter vector according to the following formula:

[0203]

[0204] in, For the first updated parameter vector, Let be the Jacobian matrix, W be the weight matrix, T be the matrix transpose operation, I be the identity matrix, λ be the regularization parameter, and y(0) be the first residual vector.

[0205] In one embodiment, the template image pixel filtering module further includes:

[0206] The template image pixel filtering unit is used to filter the template image pixels using the following formula:

[0207]

[0208] Wherein, G is the set of pixel indices, and i is the pixel index of the template image. The gradient of the i-th pixel of the template image in the x-direction is the square of the gradient. q is the square of the gradient of the i-th pixel of the template image in the y-direction, g is the gradient threshold, and q is the total number of pixels in the template image.

[0209] In one embodiment, the image positioning device for the planar object further includes:

[0210] The final image localization module is used to perform image mapping on the template image using the final homography matrix to obtain the final mapped image. The template image is the image of a planar object in the world coordinate system, and the final mapped image is the image of the planar object in the image coordinate system.

[0211] In one embodiment, the first homography matrix update module 60 further includes:

[0212] The first incremental homography matrix calculation unit is used to calculate the first incremental homography matrix corresponding to the first update parameter vector;

[0213] The first updated homography matrix calculation unit is used to take the product of the first incremental homography matrix and the homography matrix as the first updated homography matrix.

[0214] Reference Figure 7 This application also provides a computer device whose internal structure can be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor is designed to provide computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores operating devices, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores a first update parameter vector and a first update homography matrix, etc. The network interface is used for communication with external terminals via a network connection. Furthermore, the computer device may also include input devices and a display screen, etc. When executed by a processor, this computer program implements an image localization method for planar objects, comprising the following steps: obtaining a template image, a homography matrix, and a parameter vector corresponding to the homography matrix; mapping the template image using the homography matrix to obtain a first mapped image; calculating a first residual vector between the first mapped image and a current target image; calculating a weight matrix and a Jacobian matrix of the parameter vector based on the first residual vector; updating the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain a first updated parameter vector; updating the homography matrix using the first updated parameter vector to obtain a first updated homography matrix; determining whether the first updated parameter vector is less than or equal to a parameter vector threshold, and if so, using the first updated homography matrix as the final homography matrix; the final homography matrix is ​​used to localize the template image. Those skilled in the art will understand that... Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0215] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an image localization method for a planar object, comprising the following steps: obtaining a template image, a homography matrix, and a parameter vector corresponding to the homography matrix; performing image mapping on the template image using the homography matrix to obtain a first mapped image; calculating a first residual vector between the first mapped image and a current target image; calculating a weight matrix and a Jacobian matrix of the parameter vector based on the first residual vector; updating the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain a first updated parameter vector; updating the homography matrix using the first updated parameter vector to obtain a first updated homography matrix; determining whether the first updated parameter vector is less than or equal to a parameter vector threshold; if so, using the first updated homography matrix as the final homography matrix; the final homography matrix is ​​used to localize the template image.

[0216] It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0217] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0218] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0219] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for image localization of a planar object, characterized in that, include: Obtain the template image, the homography matrix, and the parameter vector corresponding to the homography matrix; The template image is mapped using the homography matrix to obtain a first mapped image; Calculate the first residual vector between the first mapped image and the current target image; Calculate the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector; The parameter vector is updated using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain the first updated parameter vector. The homography matrix is ​​updated using the first update parameter vector to obtain the first updated homography matrix; Determine whether the first updated parameter vector is less than or equal to the parameter vector threshold. If so, use the first updated homography matrix as the final homography matrix. The final homography matrix is ​​used to locate the template image.

2. The image positioning method for a planar object according to claim 1, characterized in that, The step of calculating the Jacobian matrix of the weight matrix and the parameter vector based on the first residual vector includes: Calculate the first-order partial derivative of the first residual vector with respect to the basis coefficients of the vector space to obtain the first intermediate result; Based on the first intermediate result, the Jacobian matrix of the parameter vector is calculated according to the following formula: in, Let be the Jacobian matrix of the parameter vector. J(0) is the first intermediate result, and J(0) is the second intermediate result. The difference between the first residual vector and the median of the first residual vector is used as the standardized first residual vector; Calculate the absolute median difference of the standardized first residual vector; Calculate the standard deviation of the standardized first residual vector based on the absolute median; The weight matrix is ​​calculated based on the absolute median difference and the standard deviation of the standardized first residual vector.

3. The image positioning method for a planar object according to claim 1, characterized in that, Before calculating the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector, the method further includes: Calculate the image gradient of the template image; The template image pixels are filtered according to the image gradient to obtain a pixel index set; The template image and the current target image are normalized based on the number of pixel indices in the pixel index set.

4. The image positioning method for a planar object according to claim 1, characterized in that, After determining whether the first updated parameter vector is less than or equal to the parameter vector threshold, the method further includes: If the first updated parameter vector is greater than the parameter vector threshold, then the current target image is normalized according to the first updated homography matrix; The template image is mapped using the first updated homography matrix to obtain a second mapped image; Calculate the second residual vector between the second mapped image and the current target image; Calculate the Jacobian matrix of the weight matrix and the parameter vector based on the second residual vector; The parameter vector is updated using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the second residual vector to obtain the second updated parameter vector. The homography matrix is ​​updated using the second update parameter vector to obtain the second updated homography matrix; Determine whether the second updated parameter vector is less than or equal to the parameter vector threshold. If so, use the second updated homography matrix as the final homography matrix.

5. The image positioning method for a planar object according to claim 1, characterized in that, The step of using the homography matrix to perform image mapping on the template image to obtain a first mapped image includes: Multiply the homography matrix by the pixel coordinates of each template image to obtain the mapped pixel coordinates; The mapped pixel value is determined based on the mapped pixel coordinates; The first mapped image is determined based on the mapped pixel coordinates and the mapped pixel values.

6. The image positioning method for a planar object according to claim 1, characterized in that, The step of updating the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix, and the first residual vector to obtain the first updated parameter vector includes: The first updated parameter vector is calculated according to the following formula: in, For the first updated parameter vector, Let be the Jacobian matrix, W be the weight matrix, T be the matrix transpose operation, I be the identity matrix, λ be the regularization parameter, and y(0) be the first residual vector.

7. The image positioning method for a planar object according to claim 3, characterized in that, The step of filtering template image pixels based on the image gradient to obtain a pixel index set includes: The template image pixels are filtered using the following formula: Wherein, G is the set of pixel indices, and i is the pixel index of the template image. The square of the gradient of the i-th pixel of the template image in the x-direction. q is the square of the gradient of the i-th pixel of the template image in the y-direction, g is the gradient threshold, and q is the total number of pixels in the template image.

8. The image positioning method for a planar object according to claim 1, characterized in that, After using the first updated homography matrix as the final homography matrix, the method further includes: The template image is mapped using the final homography matrix to obtain the final mapped image. The template image is the image of a planar object in the world coordinate system, and the final mapped image is the image of the planar object in the image coordinate system.

9. The image positioning method for a planar object according to claim 1, characterized in that, The step of updating the homography matrix using the first update parameter vector to obtain the first updated homography matrix includes: Calculate the first incremental homography matrix corresponding to the first updated parameter vector; The product of the first incremental homography matrix and the homography matrix is ​​used as the first updated homography matrix.

10. An image positioning device for a planar object, characterized in that, include: The parameter acquisition module is used to acquire the template image, the homography matrix, and the parameter vector corresponding to the homography matrix; The first image mapping module is used to perform image mapping on the template image using the homography matrix to obtain a first mapped image; The first residual vector calculation module is used to calculate the first residual vector between the first mapped image and the current target image; The first matrix calculation module is used to calculate the weight matrix and the Jacobian matrix of the parameter vector based on the first residual vector. The first parameter vector update module is used to update the parameter vector using Tikhonov regularization based on the Jacobian matrix, the weight matrix and the first residual vector to obtain the first updated parameter vector. The first homography matrix update module is used to update the homography matrix using the first update parameter vector to obtain the first updated homography matrix; The first determination module is used to determine whether the first updated parameter vector is less than or equal to the parameter vector threshold. If so, the first updated homography matrix is ​​used as the final homography matrix. The final homography matrix is ​​used to locate the template image.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image positioning method for a planar object as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image positioning method for a planar object as described in any one of claims 1 to 9.

Images