Image template matching method, device, electronic device and storage medium

Abstract

Embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for image template matching. The method comprises: obtaining a template image and an image to be matched, and obtaining feature information of a target template feature point; concurrently calculating the similarity between the target template feature point and a plurality of corresponding target image points based on the feature information; and adding the similarity between each target image point and the target template feature point to the comprehensive similarity corresponding to the region to be matched where the target image point is located; and determining a target matching region from each region to be matched based on the comprehensive similarity corresponding to each region to be matched. This method enables parallel calculation of the similarity between the target template feature point and target image points corresponding to a plurality of different regions in the image to be matched, thereby improving the template matching speed.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image template matching method, device, electronic equipment and storage medium. Background Art

[0002] Image template matching involves traversing the image to be matched and finding areas within it that are highly similar to a pre-generated template image. Image template matching technology is widely used in production and life, for example, in license plate recognition, barcode recognition, and character recognition.

[0003] Traditional template matching methods include: extracting template feature information of the template image, traversing different image areas of the image to be matched, determining the similarity between the image information of each image area and the template feature information one by one, and then selecting the image area with higher similarity in the image to be matched as the target area for matching.

[0004] However, the distribution position of the template feature information on the template image is discrete and disordered. Assuming that the template size is 7x7, the template feature points are 21 discrete points in the template image, and the information of the template feature points is the template feature information. Therefore, the distribution position of the template feature information on the template image is discrete and disordered.

[0005] In the process of determining the similarity between the image information of each image region of the image to be matched and the template feature information one by one, since the distribution position of the template feature information on the template image is discrete and disordered, each time the image region of the image to be matched and the template image are matched, the information of the image points corresponding to the template feature points in the current image region of the image to be matched can only be obtained one by one from the memory and matched with the template feature points until the template feature information of all template feature points is matched with the information of the corresponding image points in the image region. Then, the information of the image points corresponding to the template feature points in the next image region of the image to be matched is obtained one by one from the memory and matched again until the template feature information is matched with the image information of each image region of the image to be matched. However, this method of obtaining the information of the image points corresponding to the template feature points in each image region of the image to be matched one by one requires that when the template image is matched with each image region of the image to be matched, the image point information of the image region to be matched corresponding to each template feature point needs to be obtained one by one from the memory, which results in a slow template matching speed. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a method, device, electronic device and storage medium for image template matching to improve the template matching speed.

[0007] In a first aspect, an embodiment of the present invention provides a template matching method for an image, comprising:

[0008] Get the template image and the image to be matched;

[0009] Acquire feature information of a target template feature point of the template image, where the target template feature point is a template feature point of the template image acquired according to a preset rule;

[0010] Based on the feature information, similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point are calculated in parallel, wherein the plurality of target image points are image points in the same row or column in the image to be matched, and two adjacent target image points are separated by a preset number of columns or rows;

[0011] Accumulating the similarity between each target image point and the target template feature point to the comprehensive similarity corresponding to the to-be-matched region where the target image point is located, wherein the to-be-matched region where each target image point is located is: the region in the to-be-matched image corresponding to the template image when the target image point coincides with the target template feature point;

[0012] According to the comprehensive similarity corresponding to each to-be-matched region, a target matching region is determined from each to-be-matched region.

[0013] Optionally, before determining a target matching area from each to-be-matched area according to the comprehensive similarity corresponding to each to-be-matched area, the method further includes:

[0014] The next target template feature point of the template image is obtained according to the preset rule, and the process returns to the step of obtaining feature information of the target template feature point of the template image until all template feature points of the template image are obtained.

[0015] Optionally, after all template feature points of the template image are acquired, the method further includes:

[0016] moving the template image by a preset number of rows in a first moving direction or by a preset number of columns in a second moving direction, and returning to the step of calculating, in parallel, the similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point based on the feature information, until the template image moves to coincide with a first boundary or a second boundary of the image to be matched;

[0017] The first boundary is a boundary in the boundary of the image to be matched that corresponds to the first moving direction, and the second boundary is a boundary in the boundary of the image to be matched that corresponds to the second moving direction.

[0018] Optionally, before the similarities between the target template feature points and the target template feature points in parallel based on the feature information are calculated, the method further includes:

[0019] Based on single instruction multiple data (SIMD), loading image information of multiple target image points corresponding to the target template feature point in the image to be matched from a cache memory;

[0020] The step of calculating, in parallel based on the feature information, similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point includes:

[0021] Based on the image information and the feature information, similarities between each target image point and the target template feature point are calculated in parallel.

[0022] Optionally, the storage order of the feature information of each template feature point in the template image in the memory is consistent with the distribution order of the row and column where each template feature point is located in the template image;

[0023] The acquiring feature information of the target template feature points of the template image includes:

[0024] The feature information of the target template feature point is obtained according to the storage order of the feature information of each template feature point of the template image in the memory.

[0025] Optionally, the feature information of each template feature point includes a template offset value and image feature information of the template feature point;

[0026] The step of calculating, in parallel based on the feature information, similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point includes:

[0027] Determining a plurality of target image points corresponding to the target template feature point in the image to be matched based on a template offset value of the target template feature point, position information of an image point corresponding to a reference point of the template image in the image to be matched, a column width of the image to be matched, and the preset number of columns or rows;

[0028] The similarity between each target image point and the target template feature point is calculated in parallel based on the image feature information of the target template feature point and the image information of the multiple target image points.

[0029] Optionally, determining the plurality of target image points corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, position information of the image point corresponding to the reference point of the template image in the image to be matched, the column width of the image to be matched, and the preset number of columns or rows includes:

[0030] Based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched, the target image point currently corresponding to the target template feature point in the image to be matched is determined using the following formula:

[0031] p=i*w+j+offset

[0032] Wherein, p is the target image point currently corresponding to the target template feature point in the image to be matched, i is the ordinal number of the row in which the image point corresponding to the reference point of the template image is located in the image to be matched, w is the column width or row width of the image to be matched, j is the ordinal number of the column in which the image point corresponding to the reference point of the template image is located in the image to be matched, and offset is the template offset value of the target template feature point;

[0033] The template image is moved a preset number of columns in a third moving direction or a preset number of rows in a fourth moving direction, and the template offset value according to the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched are returned, and the step of determining the target image point currently corresponding to the target template feature point in the image to be matched using the following formula is performed until the template image moves to coincide with the third boundary or the fourth boundary of the image to be matched, wherein the third boundary is the boundary in the boundary of the image to be matched corresponding to the second moving direction, and the fourth boundary is the boundary in the boundary of the image to be matched corresponding to the fourth moving direction.

[0034] In a second aspect, an embodiment of the present invention provides an image template matching device, comprising:

[0035] An image acquisition module is used to acquire a template image and an image to be matched;

[0036] A feature information acquisition module, used to acquire feature information of target template feature points of the template image;

[0037] a similarity calculation module, configured to calculate, in parallel, the similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point based on the feature information, wherein the plurality of target image points are image points in the same row or column in the image to be matched, and two adjacent target image points are separated by a preset number of columns or rows;

[0038] a similarity accumulation module, configured to accumulate the similarity between each target image point and the target template feature point to the comprehensive similarity corresponding to the to-be-matched region where the target image point is located, wherein the to-be-matched region where each target image point is located is: the region in the to-be-matched image corresponding to the template image when the target image point coincides with the target template feature point;

[0039] The target region determination module is used to determine a target matching region from each to-be-matched region according to the comprehensive similarity corresponding to each to-be-matched region.

[0040] Optionally, the device further includes:

[0041] The feature point acquisition module is used to obtain the next target template feature point of the template image according to the preset rules before determining the target matching area from each area to be matched based on the comprehensive similarity corresponding to each area to be matched, and return to the step of obtaining the feature information of the target template feature point of the template image until all template feature points of the template image are obtained.

[0042] Optionally, the device further includes:

[0043] A moving module is used to move the template image by a preset number of rows in a first moving direction or by a preset number of columns in a second moving direction after all template feature points of the template image have been acquired, and return to the step of calculating in parallel the similarity between the target template feature points and the target template feature points corresponding to the multiple target image points in the image to be matched based on the feature information, until the template image moves to coincide with the first boundary or the second boundary of the image to be matched; wherein the first boundary is the boundary in the boundary of the image to be matched corresponding to the first moving direction, and the second boundary is the boundary in the boundary of the image to be matched corresponding to the second moving direction.

[0044] Optionally, the device further includes:

[0045] An information loading module, configured to load image information of a plurality of target image points corresponding to the target template feature points in the image to be matched from a cache memory based on a single instruction stream multiple data stream (SIMD);

[0046] The similarity calculation module is specifically configured to calculate the similarity between each target image point and the target template feature point in parallel based on the image information and the feature information.

[0047] Optionally, the storage order of the feature information of each template feature point in the template image in the memory is consistent with the distribution order of the row and column where each template feature point is located in the template image;

[0048] The feature information acquisition module is specifically configured to acquire the feature information of the target template feature point according to the storage order of the feature information of each template feature point of the template image in the memory.

[0049] Optionally, the feature information of each template feature point includes a template offset value and image feature information of the template feature point;

[0050] The similarity calculation module is specifically used to determine the multiple target image points corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, the column width of the image to be matched and the preset number of columns or rows; and calculate the similarity between each target image point and the target template feature point in parallel based on the image feature information of the target template feature point and the image information of the multiple target image points.

[0051] Optionally, the similarity calculation module includes:

[0052] The similarity calculation submodule is used to determine the target image point currently corresponding to the target template feature point in the image to be matched using the following formula based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched:

[0053] p=i*w+j+offset

[0054] Wherein, p is the target image point currently corresponding to the target template feature point in the image to be matched, i is the ordinal number of the row in which the image point corresponding to the reference point of the template image is located in the image to be matched, w is the column width or row width of the image to be matched, j is the ordinal number of the column in which the image point corresponding to the reference point of the template image is located in the image to be matched, and offset is the template offset value of the target template feature point;

[0055] A moving submodule is used to move the template image by a preset number of columns in a third moving direction or a preset number of rows in a fourth moving direction, and return the template offset value based on the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched, and use the following formula to determine the target image point currently corresponding to the target template feature point in the image to be matched, until the template image moves to coincide with the third boundary or the fourth boundary of the image to be matched, wherein the third boundary is the boundary in the boundary of the image to be matched corresponding to the second moving direction, and the fourth boundary is the boundary in the boundary of the image to be matched corresponding to the fourth moving direction.

[0056] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0057] Memory for storing computer programs;

[0058] The processor is configured to implement any of the method steps described in the first aspect when executing a program stored in the memory.

[0059] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0060] Beneficial effects of the embodiments of the present invention:

[0061] Using the method provided by an embodiment of the present invention, after each acquisition of feature information for a target template feature point in a template image, similarities between the target template feature point and multiple target image points corresponding to the target template feature point in the image to be matched can be calculated in parallel based on the feature information. That is, the acquired feature information for a template feature point can be simultaneously calculated with the information of target image points corresponding to the template feature point in multiple different image regions of the image to be matched. Therefore, data for each image point in the same row or column of the image to be matched can be loaded from memory simultaneously based on a preset number of columns or rows. When calculating the similarities between the target template feature point and the multiple target image points corresponding to the image to be matched, not only is it no longer necessary to acquire the template feature information one by one, but the similarities between the loaded multiple target image points and the target template feature point can also be calculated in parallel for each preset number of columns or rows. This allows for parallel calculation of the similarities between the target template feature point and the target image points corresponding to multiple different regions of the image to be matched, effectively improving the template matching speed.

[0062] Of course, it is not necessary to achieve all of the advantages described above simultaneously in order to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0064] Figure 1 A schematic diagram of the distribution of template feature information on a template image provided by an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of matching template feature information with the image to be matched;

[0066] Figure 3 A flow chart of an image template matching method provided by an embodiment of the present invention;

[0067] Figure 4 Another flow chart of the image template matching method provided by an embodiment of the present invention;

[0068] Figure 5 A schematic diagram of matching a template image and an image to be matched provided by an embodiment of the present invention;

[0069] Figure 6 A schematic diagram of template matching using parallel data processing provided by an embodiment of the present invention;

[0070] Figure 7 A schematic diagram of storing template feature information in a memory according to an embodiment of the present invention;

[0071] Figure 8 A schematic structural diagram of an image template matching device provided by an embodiment of the present invention;

[0072] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of the present invention.

[0074] Figure 1 A schematic diagram of the distribution of template feature information on a template image provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the size of the template image 101 is 7x7, the template feature points of the template image are 0-21, and the information corresponding to the template feature points 0-21 is the template feature information, which is represented by Figure 1 It can be seen that the distribution position of the template feature information on the template image 101 is discrete and disordered. Figure 2 A schematic diagram of matching template feature information with the image to be matched, such as Figure 2 As shown, the template image 101 is mainly matched with the image to be matched 201 to find out whether there is an area in the image to be matched 201 that matches the template image 101. Since the distribution position of the template feature information on the template image is discrete and disordered, the traditional template matching method can only obtain the information of the image points corresponding to the template feature points in the current image area of ​​the image to be matched from the memory one by one, and match it with the template feature point information until the template feature information of all template feature points is matched with the information of the corresponding image points in the image area, and then obtain the information of the image points corresponding to the template feature points in the next image area of ​​the image to be matched from the memory one by one, and match it with the template feature point information until the image information in each image area of ​​the image to be matched is matched with the template feature information. For example, Figure 2 When the template image 101 shown is matched with the image area 202 of the image to be matched that is covered by it, the feature information of the template feature point 0 is first obtained, and the similarity between the feature information of the template feature point 0 and the image information of the image point corresponding to the feature point 0 in the image area 202 is calculated. Then, the feature information of the feature point 1 is obtained, and the similarity between the feature information of the feature point 1 and the image information of the image point corresponding to the feature point 1 in the image area 202 is calculated. In this way, each piece of template feature information is obtained one by one and matched with the image information in the image area 202 until all the template feature information and the image information in the image area 202 are matched. Then, the template image is moved horizontally (or vertically) by one row (column) or multiple rows (columns) to obtain the next image area of ​​the image to be matched 201 covered by the template image 101. Then, in the same manner, each piece of template feature information is obtained one by one and matched with the image information in the image area until all the template feature information and the image information in the image area are matched. In this way, until the template feature information of the template image 101 is matched with the image information in each image region of the image to be matched 201 , the matching degree between each image region of the image to be matched 201 and the template image 101 is obtained.

[0075] However, matching the template image with each image region of the image to be matched requires loading template feature information one by one, resulting in slow template matching. This method of acquiring image point information in the image region to be matched corresponding to each template feature point and matching it with the template feature information requires each image region to be matched with the template image in turn. Furthermore, when acquiring image point information for each image region to be matched from memory, due to the mapping relationship between memory and cache, acquiring image point information for adjacent image regions to be matched can result in numerous cache misses. Consequently, the image point information for the image regions to be matched must be repeatedly loaded from memory into the cache, further slowing template matching. Cache is a temporary storage device located within the CPU processor between the CPU and external memory. Its capacity is smaller than that of memory, but its exchange rate is faster than that of memory. A cache miss occurs when the CPU cannot find the required data in the cache, requiring repeated loading of the image point information.

[0076] In order to improve the template matching speed, embodiments of the present invention provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for image template matching.

[0077] The following first introduces the image template matching method provided by the embodiment of the present invention. The image template matching method provided by the embodiment of the present invention can be applied to any electronic device with image processing function, and is not specifically limited here.

[0078] Figure 3 A flow chart of the image template matching method provided by the embodiment of the present invention is as follows: Figure 3 As shown, the method includes:

[0079] S301: Acquire a template image and an image to be matched.

[0080] In the embodiment of the present invention, the template image is a target image pre-generated for a specific application scenario. For example, in a license plate matching scenario, the template image is a target license plate image; in a face matching scenario, the template image is a face feature image, such as a face image including a face outline and / or facial features. The template image includes multiple template feature points, such as Figure 1 As shown, the template image 101 includes template feature points 0-21.

[0081] The image to be matched is the image that needs to be matched with the template image.

[0082] S302: Acquire feature information of target template feature points of the template image.

[0083] The target template feature points are template feature points of the template image obtained according to preset rules.

[0084] For example, according to the row-column distribution order of the template feature points of the template image, the first template feature point in the template image can be used as the starting point, the first template feature point can be used as the target template feature point, and the feature information of the target template feature point can be obtained. Alternatively, the last template feature point in the template image can be used as the starting point, the last template feature point can be used as the target template feature point, and the feature information of the target template feature point can be obtained.

[0085] S303 : Based on the feature information, similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point are calculated in parallel.

[0086] The multiple target image points are image points in the same row or column of the image to be matched, and adjacent target image points are separated by a preset number of columns or rows. Furthermore, when the multiple target image points are image points in the same row of the image to be matched, adjacent target image points are separated by a preset number of columns; and when the multiple target image points are image points in the same column of the image to be matched, adjacent target image points are separated by a preset number of rows.

[0087] The preset number of columns and the preset number of rows can be set according to the actual application scenario. For example, the preset number of columns can be set to 2 columns or 3 columns, and the preset number of rows can be set to 3 rows or 4 rows.

[0088] S304: Accumulate the similarity between each target image point and the target template feature point to the comprehensive similarity corresponding to the to-be-matched area where the target image point is located.

[0089] The to-be-matched region where each target image point is located is: when the target image point coincides with the target template feature point, the region in the to-be-matched image that corresponds to the template image.

[0090] S305 : Determine a target matching area from each of the to-be-matched areas according to the comprehensive similarity corresponding to each to-be-matched area.

[0091] Specifically, the image region to be matched whose corresponding comprehensive similarity is greater than a preset similarity threshold can be selected as the target matching region, wherein the preset similarity threshold can be specifically set according to the actual application scenario.

[0092] Using the method provided by an embodiment of the present invention, after each acquisition of feature information for a target template feature point in a template image, similarities between the target template feature point and multiple target image points corresponding to the target template feature point in the image to be matched can be calculated in parallel based on the feature information. That is, the acquired feature information for a template feature point can be simultaneously calculated with the information of target image points corresponding to the template feature point in multiple different image regions of the image to be matched. Therefore, data for each image point in the same row or column of the image to be matched can be loaded from memory simultaneously based on a preset number of columns or rows. When calculating the similarities between the target template feature point and the multiple target image points corresponding to the image to be matched, not only is it no longer necessary to acquire the template feature information one by one, but the similarities between the loaded multiple target image points and the target template feature point can also be calculated in parallel for each preset number of columns or rows. This allows for parallel calculation of the similarities between the target template feature point and the target image points corresponding to multiple different regions of the image to be matched, effectively improving the template matching speed.

[0093] In a possible implementation, before determining the target matching area from each to-be-matched area based on the comprehensive similarity corresponding to each to-be-matched area, the method may further include step A1:

[0094] Step A1: Acquire the next target template feature point of the template image according to the preset rule, and return to the step of acquiring feature information of the target template feature point of the template image until all template feature points of the template image are acquired.

[0095] For example, Figure 1 For example, according to the row-column distribution order of the template feature points of the template image, with the first template feature point "0" in the template image as the starting point, the first template feature point "0" is used as the target template feature point. After executing the above steps S302-S304 for the target template feature point, the next template feature point "3" of the target template feature point can be used as the next target template feature point of the template image, and then the process returns to step S302 for the next target template feature point. This process continues until all template feature points 0-21 of the template image are acquired.

[0096] Or, still Figure 1For example, in the template image in the reverse order of the row-column distribution order of the template feature points of the template image, starting from the last template feature point "12" in the template image, the first template feature point "12" is used as the target template feature point. After executing the above steps S302-S304 for the target template feature point, the next template feature point "11" of the target template feature point can be used as the next target template feature point of the template image, and then the process returns to step S302 for the next target template feature point. This process continues until all template feature points 0-21 of the template image are acquired.

[0097] In a possible implementation, after all template feature points of the template image are acquired, the method may further include step B1:

[0098] Step B1, moving the template image by a preset number of rows in a first moving direction or by a preset number of columns in a second moving direction, and returning to the step of calculating in parallel the similarity between the target template feature points and the target template feature points corresponding to the multiple target image points in the image to be matched based on the feature information, until the template image moves to coincide with the first boundary or the second boundary of the image to be matched.

[0099] The first boundary is a boundary in the boundary of the image to be matched that corresponds to the first moving direction, and the second boundary is a boundary in the boundary of the image to be matched that corresponds to the second moving direction.

[0100] If the first moving direction is from top to bottom, the first boundary is the lower boundary of the image to be matched; if the first moving direction is from bottom to top, the first boundary is the upper boundary of the image to be matched; if the second moving direction is from left to right, the second boundary is the right boundary of the image to be matched; if the second moving direction is from right to left, the second boundary is the left boundary of the image to be matched.

[0101] Specifically, the template image can be moved a preset number of rows in a first moving direction, and then the step of calculating in parallel the similarities between the multiple target image points corresponding to the target template feature points in the image to be matched and the target template feature points based on the feature information is returned until the template image moves to coincide with the first boundary of the image to be matched. Alternatively, the template image can be moved a preset number of columns in a second moving direction, and then the step of calculating in parallel the similarities between the multiple target image points corresponding to the target template feature points in the image to be matched and the target template feature points is returned based on the feature information, until the template image moves to coincide with the second boundary of the image to be matched.

[0102] In this embodiment, the template feature points of the template image can be obtained in order of row-column distribution, starting from the first template feature point in the template image, to determine the target matching area. Figure 4 , Figure 4 Another flow chart of the image template matching method provided by the embodiment of the present invention is as follows: Figure 4 As shown, the method includes:

[0103] S401: Obtain a template image and an image to be matched, and set k to 1.

[0104] S402: Acquire feature information of the kth template feature point of the template image.

[0105] S403 : Based on the feature information, similarities between a plurality of target image points corresponding to the k th template feature point in the image to be matched and the k th template feature point are calculated in parallel.

[0106] The target image points are image points in the same row of the image to be matched, and two adjacent target image points are separated by a preset number of columns. The preset number of columns can be set according to actual matching conditions and is not specifically limited here.

[0107] S404 , accumulating the similarity between each target image point and the k-th template feature point to the comprehensive similarity corresponding to the to-be-matched region where the target image point is located.

[0108] The to-be-matched region where each target image point is located is: when the target image point coincides with the k-th template feature point, the region in the to-be-matched image that corresponds to the template image.

[0109] S405 , assigning a value of k plus 1 to k, and returning to the step of obtaining feature information of the kth template feature point of the template image, until k is greater than the number of the template feature points.

[0110] S406 , moving the template image by a preset number of rows in a first moving direction, and returning to the step of setting k to 1, until the template image moves to coincide with a first boundary of the image to be matched.

[0111] The first boundary is the boundary of the image to be matched that corresponds to the first movement direction. The first movement direction can be set to a vertically upward direction or a vertically downward direction along the image rows of the image to be matched, depending on the actual matching situation. The preset number of rows can be set to 1, 2, or 3, etc., depending on the actual matching situation, and is not specifically limited here.

[0112] S407 : Determine a target matching area from each of the to-be-matched areas according to the comprehensive similarity corresponding to each to-be-matched area.

[0113] Specifically, the image region to be matched whose corresponding comprehensive similarity is greater than a preset similarity threshold can be selected as the target matching region, wherein the preset similarity threshold can be specifically set according to the actual application scenario.

[0114] Using the method provided by an embodiment of the present invention, after each time feature information of the kth template feature point of a template image is acquired, similarities between the multiple target image points corresponding to the kth template feature point in the image to be matched and the kth template feature point can be calculated in parallel based on the feature information. That is, the acquired feature information of a template feature point can be simultaneously calculated with the information of the target image points corresponding to the template feature point in multiple different image regions of the image to be matched. Therefore, data of each image point in the same row of the image to be matched can be loaded from the memory simultaneously according to a preset number of columns. Then, when calculating the similarity between the kth template feature point and the multiple target image points corresponding to the image to be matched, not only is it no longer necessary to acquire the template feature information one by one, but the similarity between the loaded multiple target image points every preset number of columns and the kth template feature point can also be calculated in parallel. That is, the similarity between the kth template feature point and the target image points corresponding to multiple different regions in the image to be matched can be calculated in parallel, effectively improving the template matching speed.

[0115] In a possible implementation, the feature information of each template feature point includes the template offset value and image feature information of the template feature point. The parallel calculation of the similarity between the target template feature point and the target image points corresponding to the target template feature point in the image to be matched based on the feature information may include the following steps C1-C2

[0116] Step C1: Determine the multiple target image points corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, the column width of the image to be matched and the preset number of columns or rows.

[0117] The template offset value represents the offset of the template feature point relative to the reference point of the template image, and can specifically include the horizontal offset and vertical offset of the template feature point relative to the reference point of the template image. The reference point of the template image can be customized in actual applications, for example, Figure 1For the template image 101 shown, the reference point of the template image 101 can be set as the image point in the upper left corner of the template image 101. The image point can specifically be a pixel point of the image. Then, the horizontal offset of the template feature point 1 in the template image 101 relative to the reference point is 2, and the vertical offset is 1. Then, the offset value of the template feature point 1 can be expressed as [1, 2]. For another example, the horizontal offset of the template feature point 18 in the template image 101 relative to the reference point is 6, and the vertical offset is 2. Then, the offset value of the template feature point 18 can be expressed as [2, 6].

[0118] The image feature information of the template feature point includes at least one of pixel information, brightness information, chromaticity information, gradient information and position information of the template feature point.

[0119] Figure 5 A schematic diagram of matching a template image with an image to be matched provided by an embodiment of the present invention. Figure 5 As shown, if the image point in the upper left corner of template image 501 is set as the reference point, the image point corresponding to the reference point in image to be matched 502 is: the image point in row 0 and column 1 in image to be matched 502. The position information of the image point corresponding to the reference point of the template image in the image to be matched can specifically be the position coordinates of the image point in the image to be matched.

[0120] In a possible implementation, this step may determine the target image point currently corresponding to the target template feature point in the image to be matched using the following formula based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched:

[0121] p=i*w+j+offset

[0122] Where p is the target image point currently corresponding to the target template feature point in the image to be matched, i is the row number of the image point corresponding to the reference point of the template image in the image to be matched, w is the column width or row width of the image to be matched, j is the column number of the image point corresponding to the reference point of the template image in the image to be matched, and offset is the template offset value of the target template feature point. If the target template feature points are obtained from the template image row by row, then w is the column width of the image to be matched; if the target template feature points are obtained from the template image column by column, then w is the row width of the image to be matched.

[0123] For example, Figure 5As shown, the image to be matched 502 includes 14 rows and 18 columns of image points. The image point in the upper left corner of the template image 501 can be set as the reference point, and the image point corresponding to the reference point in the image to be matched 502 is image point a2. If the target template feature point is template feature point 1 among template feature points 0-21 of the template image 501, the offset value of template feature point 1 can be obtained as [1, 2], and the position coordinates of image point a2 in the image to be matched 502 are (0, 1), that is, the ordinal number of the row where image point a2 is located is 0 and the ordinal number of the column where it is located is 1, and the column width of the image to be matched 502 is 18. Then, substituting the above information into the formula p = i*w+j+offset = 0*18+1+(1*18+2) = (0+1)*18+(1+2) = [1, 3], it is obtained that the image point corresponding to the target template feature point in the image to be matched 502 is the image point in the 1st row and 3rd column. The index subscripts of the rows and columns in the image to be matched start from 0.

[0124] Then, the template image can be moved a preset number of columns in a third moving direction, and the step of determining the target image point currently corresponding to the target template feature point in the image to be matched using the following formula based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width of the image to be matched is returned until the template image moves to coincide with a third boundary of the image to be matched. The third boundary is the boundary of the image to be matched that corresponds to the third moving direction.

[0125] The third moving direction can be set to the horizontal right direction or the horizontal left direction along the image row of the image to be matched according to the actual matching situation. The preset number of columns can be set to 1, 2 or 3 according to the actual matching situation, and is not specifically limited here.

[0126] For example, Figure 5As shown, if the preset number of columns is set to 1, the third movement direction is set to be horizontally rightward along the image row of the image to be matched 502, and the target template feature point is template feature point 0, after determining the target image point currently corresponding to template feature point 0 in the image to be matched 502—image point a5—the template image 502 can be moved by one column along the third movement direction, and the step of determining the target image point currently corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width of the image to be matched is returned using the following formula, thereby determining the target image point currently corresponding to template feature point 0 in the image to be matched 502—image point a6. This process continues until the right boundary of the template image 502 moves to coincide with the third boundary of the image to be matched. The target image points corresponding to template feature point 0 can be obtained as: image point a5 - image point a15 in the image to be matched 502.

[0127] Let’s take another example to illustrate. Figure 5 For example, let's still set the image point in the upper left corner of template image 501 as the reference point. The image point corresponding to the reference point in the image to be matched 502 is image point a2. If the target template feature point is template feature point 1 among template feature points 0-21 of template image 501, the offset value of template feature point 1 can be obtained as [1, 2]. The position coordinates of image point a2 in the image to be matched 502 are (0, 1), that is, the ordinal number of the row where image point a2 is located is 0 and the ordinal number of the column where it is located is 1. The row width of the image to be matched 502 is 14. Substituting the above information into the formula p = i*w+j+offset = 0*14+1+(1*14+2) = (0+1)*14+(1+2) = [1, 3], the image point corresponding to the target template feature point in the image to be matched 502 is the image point in the 1st row and 3rd column. The index subscripts of the rows and columns in the image to be matched start at 0.

[0128] Then, the template image can be moved a preset number of rows in a fourth moving direction, and the step of determining the target image point currently corresponding to the target template feature point in the image to be matched using the following formula based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the row width of the image to be matched is returned until the template image moves to coincide with a fourth boundary of the image to be matched. The fourth boundary is the boundary of the image to be matched that corresponds to the fourth moving direction.

[0129] The fourth moving direction can be set to a vertically upward direction or a vertically downward direction along the image column of the image to be matched according to the actual matching situation. The preset number of rows can be set to 1, 2 or 3 according to the actual matching situation, and is not specifically limited here.

[0130] Still Figure 5 For example, if the preset number of rows is set to 1, the fourth moving direction is set to be from top to bottom along the image column of the image to be matched 502, and the target template feature point is template feature point 3, after determining the target image point b1 currently corresponding to template feature point 3 in the image to be matched, the template image 502 can be moved by one row along the fourth moving direction, and the step of determining the target image point currently corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the row width of the image to be matched is returned using the following formula, thereby determining the target image point c1 currently corresponding to template feature point 3 in the image to be matched. This step is repeated until the lower boundary of the template image 502 moves to coincide with the fourth boundary of the image to be matched. The target image points corresponding to template feature point 3 can be obtained as follows: image points b1, c1, d1, e1, f1, g1, h1, and k1 in the image to be matched 502.

[0131] Step C2: Calculate in parallel the similarity between each target image point and the target template feature point based on the image feature information of the target template feature point and the image information of the multiple target image points.

[0132] For example, Figure 5 For example, if the template image moves along the third moving direction, if the preset number of columns is set to 1, the third moving direction is set to the horizontal rightward direction along the image rows of the image to be matched 502, and the target template feature point is template feature point 0, the target image points corresponding to template feature point 0 can be obtained as: image points a5-a15 in the image to be matched 502. In this embodiment of the present invention, the similarities between image points a5-a15 and the target template feature points can be directly calculated in parallel.

[0133] Still Figure 5For example, if the template image moves along the fourth moving direction, if the preset number of rows is set to 1, the fourth moving direction is set to be along the image column of the to-be-matched image 502 from top to bottom, and the target template feature point is template feature point 3, the target image points corresponding to template feature point 3 can be obtained as: image point b1, image point c1, image point d1, image point e1, image point f1, image point g1, image point h1, and image point k1 in the to-be-matched image 502. In this embodiment of the present invention, the similarities between image point b1, image point c1, image point d1, image point e1, image point f1, image point g1, image point h1, and image point k1 and the target template feature point can be directly calculated in parallel.

[0134] The image information of each target image point corresponding to the target template feature point in the image to be matched includes at least one of pixel information, brightness information, chromaticity information, gradient information and position information of the target image point.

[0135] Specifically, this step can calculate any one of the cosine similarity, Euclidean distance and Manhattan distance between each target image point corresponding to the target template feature point in the image to be matched and the target template feature point as the similarity. Alternatively, at least two of the cosine similarity, Euclidean distance and Manhattan distance between each target image point corresponding to the target template feature point in the image to be matched and the target template feature point can be calculated, and then the average value is calculated, and the average value is used as the similarity. Of course, in the embodiment of the present invention, other similarity calculation methods can also be used based on the image feature information of the target template feature point and the image information of each target image point corresponding to the target template feature point in the image to be matched, to calculate the similarity between each target image point corresponding to the target template feature point in the image to be matched and the target template feature point, which is not specifically limited here.

[0136] The following examples are Figure 5 A specific example of template matching between the template image 501 and the image to be matched 502 is shown:

[0137] like Figure 5As shown, rows 0 through 13 of the image to be matched 502 sequentially include: image points a1-a18, image points b1-b18, ..., image points n1-n18. The reference point of template image 501 can be set to the image point in its upper left corner. The third movement direction can be set to the horizontal rightward direction from image point a1 to a18, and the third boundary is the boundary in the image to be matched 502 corresponding to the third movement direction. The first movement direction can be set to the vertical downward direction from image point a1 to n1, and the first boundary is the boundary in the image to be matched 502 corresponding to the first movement direction. The preset number of columns and the preset number of rows are set to 1.

[0138] In this example, when the matching process starts, the reference point of the template image 502 corresponds to the image point a2 in the image to be matched, and the feature information of the first template feature point (template feature point 0) can be obtained from the memory; based on the feature information of the first template feature point, multiple target image points (a5-a15) corresponding to the first template feature point in the image to be matched 502 are determined, and then the similarity between the first template feature point and the image points a5-a15 is calculated in parallel. The similarity between each target image point (image point a5-a15) and the first template feature point is added to the comprehensive similarity corresponding to the area to be matched where the target image point is located, wherein the area to be matched where each target image point (image point a5-a15) is located is: when the target image point coincides with the first template feature point, the area in the image to be matched 502 that corresponds to the template image 501. For example, Figure 5 The image point a5 shown coincides with the first template feature point, and the area in the current image 502 to be matched that corresponds to the template image 501 is the area to be matched where the image point a5 is located.

[0139] Alternatively, in this example, at the start of the matching process, the reference point of template image 502 corresponds to image point a2 in the image to be matched, and the feature information of the first template feature point (template feature point 0) can be obtained from the memory; then, based on the feature information of the first template feature point, the preset number of target image points corresponding to the first template feature point in the image to be matched 502 are first determined in sequence, and then the similarity between the first template feature point and the preset number of target image points is calculated in parallel. The preset number can be set to 8 or 10, etc., and is not specifically limited here. The number of target image points for which the similarity is calculated in parallel each time does not exceed the preset number.

[0140] For example, if the preset number is set to 8, at the start of the matching process, the reference point of template image 502 corresponds to image point a2 in the image to be matched, and the feature information of the first template feature point (template feature point 0) can be obtained from the memory. Based on the feature information of the first template feature point, the eight target image points (a5-a12) corresponding to the first template feature point in the image to be matched 502 are first determined, and then the similarities between the first template feature point and image points a5-a12 are calculated in parallel. The similarities between each image point in a5-a12 and the first template feature point are added to the comprehensive similarity corresponding to the area to be matched where the image point is located. Then, based on the feature information of template feature point 0, the other target image points (a13-a15) corresponding to template feature point 0 in the image to be matched 502 are determined, and then the similarities between template feature point 0 and image points a13-a15 are calculated in parallel. The similarity between each image point a13-a15 and the template feature point 0 is accumulated to the comprehensive similarity corresponding to the to-be-matched area where the image point is located.

[0141] Then, the feature information of the second template feature point (template feature point 1) can be obtained in the same way, and the similarity between the second template feature point and the multiple target image points corresponding to it in the image to be matched 502 can be continued to be calculated according to the above process to obtain the corresponding comprehensive similarity of each area to be matched after the new round of accumulation; for the 3rd to 22nd template feature points (template feature points 2-21), the similarity between the template feature point and the multiple target image points corresponding to it in the image to be matched 502 can be calculated according to the above process to obtain the final corresponding comprehensive similarity of each area to be matched after the accumulation of the similarities calculated for the 1st to 22nd template feature points. After all template feature points 0-21 of the template image 501 have participated in the similarity calculation, the template image 501 is moved by 1 row in the first moving direction, and the image point corresponding to the reference point of the template image 501 in the image to be matched 502 is b1. Then, according to the above process, the similarities between the 1st template feature point to the 23rd template feature point and the corresponding multiple target image points in the image to be matched 502 are calculated in sequence, until the template image moves to coincide with the first boundary of the image to be matched, and the final corresponding comprehensive similarity of each area to be matched after the similarity calculated by the 1st to 22nd template feature points is accumulated. Then, according to the comprehensive similarity corresponding to each area to be matched, the target matching area can be selected from each area to be matched. Specifically, the area to be matched with the highest comprehensive similarity in the area to be matched can be selected as the target matching area. For example, if the area in the image to be matched 501 corresponding to the template image is the area to be matched with the highest comprehensive similarity, the area can be determined as the target matching area, that is, Figure 5The area of ​​the image to be matched covered by the template image 501 is determined as the target matching area. The target matching area is the image area that ultimately corresponds to the template image.

[0142] The following examples are Figure 5 Another specific example of template matching between the template image 501 and the image to be matched 502 is shown:

[0143] like Figure 5 As shown, rows 0 through 13 of the image to be matched 502 sequentially include: image points a1-a18, image points b1-b18, ..., image points n1-n18. The reference point of template image 501 can be set to the image point in its upper left corner. The fourth movement direction can be set to the vertical downward direction from image point b1 to n1, and the fourth boundary is the boundary in the image to be matched 502 corresponding to the fourth movement direction. The second movement direction can be set to the horizontal rightward direction from image point a1 to a18, and the second boundary is the boundary in the image to be matched 502 corresponding to the second movement direction. The preset number of columns and the preset number of rows are set to 1.

[0144] In this example, when the matching process starts, the reference point of the template image 502 corresponds to the image point a2 in the image to be matched, and the feature information of the first template feature point (template feature point 0) can be obtained from the memory; based on the feature information of the first template feature point, multiple target image points (image points a5, b5..., h5) corresponding to the first template feature point in the image to be matched 502 are determined, and then the similarity between the first template feature point and the "image points a5, b5..., h5" is calculated in parallel. The similarity between each target image point (image point a5, b5..., h5) and the first template feature point is added to the comprehensive similarity corresponding to the area to be matched where the target image point is located, wherein the area to be matched where each target image point (image point a5, b5..., h5) is located is: when the target image point coincides with the first template feature point, the area in the image to be matched 502 that corresponds to the template image 501. For example, Figure 5 The image point a5 shown coincides with the first template feature point, and the area in the current image 502 to be matched that corresponds to the template image 501 is the area to be matched where the image point a5 is located.

[0145] Alternatively, in this example, at the beginning of the matching process, the reference point of the template image 502 corresponds to the image point a2 in the image to be matched, and the feature information of the first template feature point (template feature point 0) can be obtained from the memory; then, based on the feature information of the first template feature point, the preset number of target image points corresponding to the first template feature point in the image to be matched 502 are determined in sequence, and then the similarity between the first template feature point and the preset number of target image points is calculated in parallel, and the number of target image points for which the similarity is calculated in parallel each time does not exceed the preset number.

[0146] For example, if the preset number is set to 6, at the start of the matching process, the reference point of template image 502 corresponds to image point a2 in the image to be matched. The feature information of the first template feature point (template feature point 0) can be retrieved from the memory. Based on the feature information of the first template feature point, the six target image points (image points a5, b5, ..., f5) corresponding to the first template feature point in the image to be matched 502 are first determined. Then, the similarities between the first template feature point and "image points a5, b5, ..., f5" are calculated in parallel. The similarities between each image point in "image points a5, b5, ..., f5" and the first template feature point are added to the comprehensive similarity corresponding to the area to be matched where the image point is located. Then, based on the feature information of template feature point 0, the other target image points (g5 and k5) corresponding to template feature point 0 in the image to be matched 502 are determined. Then, the similarities between template feature point 0 and image points g5 and k5 are calculated in parallel. The similarity between each image point in g5 and k5 and the template feature point 0 is accumulated to the comprehensive similarity corresponding to the to-be-matched area where the image point is located.

[0147] Then, the feature information of the second template feature point (template feature point 1) can be obtained in the same way, and the similarity between the second template feature point and the multiple target image points corresponding to it in the image to be matched 502 can be continued to be calculated according to the above process to obtain the corresponding comprehensive similarity of each area to be matched after the new round of accumulation; for the 3rd to 22nd template feature points (template feature points 2-21), the similarity between the template feature point and the multiple target image points corresponding to it in the image to be matched 502 can be calculated according to the above process to obtain the final corresponding comprehensive similarity of each area to be matched after the accumulation of the similarities calculated for the 1st to 22nd template feature points. After all template feature points 0-21 of the template image 501 have participated in the similarity calculation, the template image 501 is moved by 1 column in the second moving direction, and the image point a3 corresponding to the reference point of the template image 501 in the image to be matched 502 is obtained. Then, according to the above process, the similarities between the 1st template feature point to the 23rd template feature point and the corresponding multiple target image points in the image to be matched 502 are calculated in sequence, until the template image moves to coincide with the second boundary of the image to be matched, and the final corresponding comprehensive similarity of each area to be matched after the similarity calculated by the 1st to 22nd template feature points is obtained. Then, according to the comprehensive similarity corresponding to each area to be matched, the target matching area can be selected from each area to be matched. Specifically, the area to be matched with the highest comprehensive similarity in the area to be matched can be selected as the target matching area. For example, if the area in the image to be matched 501 corresponding to the template image is the area to be matched with the highest comprehensive similarity, the area can be determined as the target matching area, that is, Figure 5 The area of ​​the image to be matched covered by the template image 501 is determined as the target matching area. The target matching area is the image area that ultimately corresponds to the template image.

[0148] In a possible embodiment, before the similarity between the target template feature point and multiple target image points corresponding to the target template feature point in the image to be matched is calculated in parallel based on the feature information, it also includes: based on SIMD (Single Instruction Multiple Data), loading the image information of the multiple target image points corresponding to the target template feature point in the image to be matched from a cache.

[0149] In an embodiment of the present invention, image information for each image point in the row corresponding to the target template feature point in the image to be matched can be pre-loaded from memory into a vector register, and the image information can be cached in a high-speed buffer. SIMD is a parallel processing method that uses a single computer instruction to perform multiple data operations, and the high-speed buffer can be a cache, for example.

[0150] The parallel calculation of the similarity between the target template feature point and a plurality of target image points corresponding to the target template feature point in the image to be matched based on the feature information may specifically include: parallel calculation of the similarity between each target image point and the target template feature point based on the image information and the feature information.

[0151] Specifically, the image information of multiple target image points corresponding to the target template feature point in the image to be matched can be loaded from the cache into a vector register for parallel calculation of the similarity between each target image point and the target template feature point.

[0152] In an embodiment of the present invention, since the template can be moved along the third moving direction by a preset number of columns or the template can be moved along the fourth moving direction by a preset number of rows, the preset number of columns and the preset number of rows are known, and the image points corresponding to the target template feature points in the image to be matched can also be known in advance when the matching process starts. Then, according to the preset number of columns and the preset number of rows, at the beginning of the matching process, the image information of multiple target image points corresponding to the target template feature points in the image to be matched can be loaded from the cache into the vector register based on SIMD, and then the similarity between the target template feature points and each target image point is calculated in parallel.

[0153] For example, Figure 5 As shown, if the preset number of columns is set to 1, and the similarity between template feature point 0 and image points a5-a15 in the image to be matched 502 is determined to be calculated sequentially based on the preset number of columns, then at the beginning of the matching process, the image information of each image point in the row (row 0) corresponding to the image point of template feature point 0 in the image to be matched is loaded from the cache based on SIMD, and then the similarity between template feature point 0 and image points a5-a15 is calculated in parallel. Alternatively, if the preset number of rows is set to 1, and the similarity between template feature point 0 and "image points a5, b5, ..., h5" in the image to be matched 502 is determined to be calculated sequentially based on the preset number of rows, then at the beginning of the matching process, the image information of each image point in the column (column 5) corresponding to the image point of template feature point 0 in the image to be matched is loaded from the cache based on SIMD, and then the similarity between template feature point 0 and "image points a5, b5, ..., h5" is calculated in parallel.

[0154] For another example, Figure 6 A schematic diagram of template matching using parallel data processing provided by an embodiment of the present invention. Figure 6 (1) is the template image 601. The distribution of each template feature point 0-21 in the template image 601 is discrete and disordered. For example, the distribution of the template feature points 0-21 in the memory is as follows: Figure 6 As shown in the memory distribution diagram 602, the distribution order of each template feature point in the memory is 0-21. The feature information of each template feature point can be taken out from the memory and put into the cache, such as Figure 6 As shown, the feature information of template feature point 0 can be retrieved from the memory and placed in a cache. When matching begins, the preset number of columns and the preset number of rows can be set to 1. The multiple target image points corresponding to template feature point 0 in the image to be matched 603 are all located in row 0 of the image to be matched 603. Based on the preset number of columns, it is determined that the similarity between template feature point 0 and image points 3-18 in row 0 of the image to be matched 603 will be calculated sequentially. At the beginning of the matching process, the image information of each image point in row 0 of the image to be matched can be loaded from the cache into a vector register based on SIMD. The feature information of template feature point 0 can also be retrieved from the cache and placed in the vector register. The similarity between template feature point 0 and image points 3-18 in row 0 of the image to be matched 602 can then be calculated in parallel. The vector register is a temporary data storage space closest to the CPU. When calculating the similarity between a template feature point and the corresponding multiple target image points, the CPU only accesses data with the vector register. Therefore, the data in the cache can be retrieved and temporarily stored in the vector register. Figure 6 The op (Over-Provision) in it refers to the cache space of the vector register.

[0155] It can be seen that in this embodiment, for the image to be matched, each time the similarity between the template feature point and the corresponding multiple target image points is calculated in a loop, since the position of the image point corresponding to the template feature point can be predicted based on the preset number of columns and the preset number of rows, SIMD technology can be used to perform data parallel processing to improve the template matching speed.

[0156] In one possible implementation, to further improve template matching speed, this implementation improves the way in which the feature information of each template feature point in the template image is stored in memory. Specifically, the order in which the feature information of each template feature point in the template image is stored in memory can be made consistent with the order in which each template feature point is distributed in the template image in its rows and columns. Prior to obtaining the feature information of the target template feature point in the template image, the method may further include obtaining the feature information of the target template feature point in accordance with the order in which the feature information of each template feature point in the template image is stored in memory.

[0157] Specifically, the feature information of each template feature point of the template image can be loaded from the memory to the cache according to the storage order of the feature information of each template feature point of the template image in the memory, and then after the matching process starts, the feature information of the target template feature point of the template image can be obtained from the cache.

[0158] Before this embodiment improves the storage method of the feature information of each template feature point in the template image in the memory, such as Figure 6 As shown in Figure 602, the memory distribution of template feature points 0-21 is stored in the memory in the order of template feature points 0-21. When performing template matching, the template feature information needs to be retrieved from the memory and placed in the cache in the order in which the template feature information is stored in the memory. The cache order is consistent with the memory order. Therefore, this memory method will result in: when the template feature information is retrieved from the cache in sequence to match the image information of the image point on the image to be matched, due to the disordered distribution position of the template feature information on the template image, when traversing the current area to be matched of the image to be matched and matching the current area to be matched with the template image, the image point information of the image to be matched that has been originally loaded into the cache is replaced by the subsequent image point information that is far away. When traversing the image point of the replaced image to be matched subsequently, the image information of the image point needs to be reloaded from the memory to the cache. As shown in FIG. Figure 6 As shown, according to Figure 6In the memory distribution diagram 602, the memory storage order is shown. When traversing the matching area of ​​the image to be matched for template feature points 3-6, the image information of the image points in rows 2-5 of the image to be matched can be sequentially retrieved from the memory and stored in the cache. When traversing the matching area of ​​the image to be matched for template feature point 9, the image information of the image points in rows 2-5 of the image to be matched may have occupied all the cache space. In this case, the image information of the image points in row 2 of the image to be matched can be replaced with the image information of the image points in row 6 of the image to be matched. Subsequently, when traversing the matching area of ​​the image to be matched for template feature point 19, since the image information of the image points in row 2 of the image to be matched has been replaced, the image information of the image points in row 2 of the image to be matched needs to be reloaded from the memory into the cache to calculate the similarity between template feature point 19 and the corresponding image points in row 2. This phenomenon in which the CPU cannot find the required data in the cache and therefore needs to repeatedly load the image information of the image point is called a cache miss. The more disordered the distribution of the template feature information of the template feature points on the template image is, the more serious the cache miss phenomenon will be, which will greatly affect the speed of template matching.

[0159] Therefore, in order to reduce cache miss and further improve the template matching speed, this embodiment makes the storage order of the feature information of each template feature point in the template image in the memory consistent with the distribution order of the row and column of each template feature point in the template image. Figure 7 A schematic diagram of storing the template feature information in the memory provided by the embodiment of the present invention, and a schematic diagram of the distribution of the template feature points 0-21 in the template image can be shown as follows: Figure 6 As shown in 601. Figure 7As shown, the storage order of the feature information of each template feature point in the template image 601 in the memory is made consistent with the distribution order of the row and column of each template feature point in the template image, that is, the feature information of each template feature point in the template image 601 is stored in the memory according to the order of the row and column of the template feature point in the template image. After this improvement, the feature information of each template feature point of the template image 601 is loaded from the memory to the cache according to the storage order of the feature information of each template feature point of the template image 601 in the memory, and the cache order of the feature information of each template feature point of the template image 601 in the cache is consistent with the storage order in the memory. When template matching is performed, the template feature information is taken out from the memory and put into the cache according to the memory storage order of the template feature information, and its cache order is consistent with the memory order. According to the improved storage method, when matching the area to be matched with the template image, the probability of the image point information of the image to be matched that has been originally loaded into the cache being replaced by the subsequent image point information that is far away can be reduced, thereby reducing cache misses and improving the template matching speed. Figure 5 As shown, according to Figure 7The order in which the template feature information is stored in memory can be determined by traversing the image to be matched for each template feature point 3-6 in the same row, and then traversing the image to be matched for each template feature point in the next row. For example, the image information of the image points in the second row of the image to be matched 502 can be first retrieved from the memory and placed in the cache. Then, the image to be matched 502 can be traversed in sequence for template feature points 3, 2, 1, 21, 20, and 19 in the same row. Then, the image to be matched 502 can be traversed in sequence for template feature points 4 and 18 in the same row, in the order of the rows. When traversing the image to be matched 502 for template feature point 9, all cache space in the cache may have been occupied by the image information of the 2nd to 5th rows of image points on the image to be matched 502. At this time, the image information of the 6th row of image points on the image to be matched 502 is used to replace the image information of the 2nd row of image points on the image to be matched 502 in the cache. There will also be no situation where, when traversing the image to be matched for template feature point 19 later, the image information of the 2nd row of image points on the image to be matched 502 has been replaced, and therefore the image information of the 2nd row of image points on the image to be matched 502 needs to be reloaded from the memory to the cache. This is because the storage order of template feature point 19 in the memory is before template feature point 9, and template feature point 19 has completed the task of traversing the image to be matched before template feature point 9. Therefore, this embodiment improves the storage method of the feature information of each template feature point in the template image in the memory, so that the storage order of the feature information of each template feature point in the template image in the memory is consistent with the distribution order of the rows and columns of each template feature point in the template image, which can reduce cache. The miss phenomenon is eliminated, and continuous memory access is achieved when traversing the image to be matched, which improves the memory access efficiency and further increases the template matching speed.

[0160] Corresponding to the above-mentioned image template matching method, an embodiment of the present invention further provides an image template matching device. The image template matching device provided by the embodiment of the present invention is introduced below. Figure 8 As shown, a template matching device for an image, the device comprising:

[0161] Image acquisition module 801, used to acquire a template image and an image to be matched;

[0162] A feature information acquisition module 802 is configured to acquire feature information of a target template feature point of the template image, wherein the target template feature point is a template feature point of the template image acquired according to a preset rule;

[0163] a similarity calculation module 803 for calculating, in parallel, the similarities between the target template feature point and a plurality of target image points corresponding to the target template feature point in the image to be matched, based on the feature information, wherein the plurality of target image points are image points in the same row or column in the image to be matched, and two adjacent target image points are separated by a preset number of columns or rows;

[0164] a similarity accumulation module 804 configured to accumulate the similarity between each target image point and the target template feature point to a comprehensive similarity corresponding to the to-be-matched region where the target image point is located, wherein the to-be-matched region where each target image point is located is the region in the to-be-matched image corresponding to the template image when the target image point coincides with the target template feature point;

[0165] The target region determination module 805 is configured to determine a target matching region from each of the regions to be matched according to the comprehensive similarity corresponding to each region to be matched.

[0166] It can be seen that, using the apparatus provided by the embodiments of the present invention, after each acquisition of feature information of a target template feature point of a template image, the similarity between the target template feature point and the target template feature point can be calculated in parallel based on the feature information. That is, the feature information of a template feature point acquired can be simultaneously calculated with the information of target image points corresponding to the template feature point in multiple different image regions of the image to be matched. Therefore, the data of each image point in the same row or column of the image to be matched can be loaded from the memory simultaneously according to a preset number of columns or rows. Then, when calculating the similarity between the target template feature point and the target image points in the image to be matched, not only is it no longer necessary to acquire the template feature information one by one, but the similarity between the loaded target image points and the target template feature point can also be calculated in parallel for each preset number of columns or rows. That is, the similarity between the target template feature point and the target image points corresponding to multiple different regions in the image to be matched can be calculated in parallel, effectively improving the template matching speed.

[0167] Optionally, the device further includes:

[0168] A feature point acquisition module (not shown in the figure) is used to obtain the next target template feature point of the template image according to the preset rules before determining the target matching area from each area to be matched based on the comprehensive similarity corresponding to each area to be matched, and return to the step of obtaining the feature information of the target template feature point of the template image until all the template feature points of the template image are acquired.

[0169] Optionally, the device further includes:

[0170] A moving module (not shown in the figure) is used to move the template image by a preset number of rows in a first moving direction or a preset number of columns in a second moving direction after all template feature points of the template image have been acquired, and return to the step of calculating in parallel the similarity between the multiple target image points corresponding to the target template feature point in the image to be matched and the target template feature point based on the feature information, until the template image moves to coincide with the first boundary or the second boundary of the image to be matched; wherein the first boundary is the boundary in the boundary of the image to be matched corresponding to the first moving direction, and the second boundary is the boundary in the boundary of the image to be matched corresponding to the second moving direction.

[0171] Optionally, the device further includes:

[0172] An information loading module (not shown in the figure) is used to load image information of multiple target image points corresponding to the target template feature points in the image to be matched from a cache memory based on single instruction stream multiple data (SIMD);

[0173] The similarity calculation module 803 is specifically configured to calculate the similarity between each target image point and the target template feature point in parallel based on the image information and the feature information.

[0174] Optionally, the storage order of the feature information of each template feature point in the template image in the memory is consistent with the distribution order of the row and column where each template feature point is located in the template image;

[0175] The feature information acquisition module 802 is specifically configured to acquire the feature information of the target template feature point according to the storage order of the feature information of each template feature point of the template image in the memory.

[0176] Optionally, the feature information of each template feature point includes a template offset value and image feature information of the template feature point;

[0177] The similarity calculation module 803 is specifically used to determine the multiple target image points corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, the column width of the image to be matched and the preset number of columns or rows; and calculate the similarity between each target image point and the target template feature point in parallel based on the image feature information of the target template feature point and the image information of the multiple target image points.

[0178] Optionally, the similarity calculation module 803 includes:

[0179] The similarity calculation submodule (not shown in the figure) is used to determine the target image point currently corresponding to the target template feature point in the image to be matched using the following formula based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched:

[0180] p=i*w+j+offset

[0181] Wherein, p is the target image point currently corresponding to the target template feature point in the image to be matched, i is the ordinal number of the row in which the image point corresponding to the reference point of the template image is located in the image to be matched, w is the column width or row width of the image to be matched, j is the ordinal number of the column in which the image point corresponding to the reference point of the template image is located in the image to be matched, and offset is the template offset value of the target template feature point;

[0182] A moving submodule (not shown in the figure) is used to move the template image by a preset number of columns in a third moving direction or a preset number of rows in a fourth moving direction, and return the template offset value based on the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched, and use the following formula to determine the target image point currently corresponding to the target template feature point in the image to be matched, until the template image moves to coincide with the third boundary or the fourth boundary of the image to be matched, wherein the third boundary is the boundary in the boundary of the image to be matched corresponding to the third moving direction, and the fourth boundary is the boundary in the boundary of the image to be matched corresponding to the fourth moving direction.

[0183] Using the apparatus provided by an embodiment of the present invention, each time feature information of a target template feature point of a template image is acquired, similarities between the target template feature point and multiple target image points corresponding to the target template feature point in the image to be matched can be calculated in parallel based on the feature information. That is, the acquired feature information of a template feature point can be simultaneously calculated with the information of target image points corresponding to the template feature point in multiple different image regions of the image to be matched. Therefore, data of each image point in the same row of the image to be matched can be loaded from the memory simultaneously according to a preset number of columns or rows. Then, when calculating the similarity between the target template feature point and the multiple target image points corresponding to the image to be matched, not only is it no longer necessary to acquire the template feature information one by one, but the similarity between the loaded multiple target image points and the target template feature point can also be calculated in parallel for each preset number of columns or rows. That is, the similarity between the target template feature point and the target image points corresponding to multiple different regions in the image to be matched can be calculated in parallel, effectively improving the template matching speed. Moreover, the embodiment of the present invention can improve the storage method of the feature information of each template feature point in the template image in the memory, so that the storage order of the feature information of each template feature point in the template image in the memory is consistent with the distribution order of the rows and columns of each template feature point in the template image, thereby reducing the cache miss phenomenon and further improving the template matching speed.

[0184] The embodiment of the present invention further provides an electronic device, such as Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0185] Memory 903, used for storing computer programs;

[0186] The processor 901 is configured to implement any of the steps of the image template matching method when executing the program stored in the memory 903 .

[0187] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0188] The communication interface is used for communication between the above electronic device and other devices.

[0189] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0190] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0191] In another embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned image template matching methods are implemented.

[0192] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the template matching method for any one of the above-mentioned embodiments.

[0193] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0194] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0195] Each embodiment in this specification is described in a related manner. Similar portions between embodiments can be referenced to each other. Each embodiment focuses on the differences between other embodiments. In particular, the system, electronic device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For related portions, reference can be made to the descriptions of the method embodiments.

[0196] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A template matching method for an image, characterized in that: include: Get the template image and the image to be matched; Acquire feature information of a target template feature point of the template image, where the target template feature point is a template feature point of the template image acquired according to a preset rule; For each target template feature point, similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point are calculated in parallel based on feature information of the target template feature point, wherein the plurality of target image points are image points in the same row or column in the image to be matched, and two adjacent target image points are separated by a preset number of columns or rows; Accumulating the similarity between each target image point and the target template feature point to the comprehensive similarity corresponding to the to-be-matched region where the target image point is located, wherein the to-be-matched region where each target image point is located is: the region in the to-be-matched image corresponding to the template image when the target image point coincides with the target template feature point; According to the comprehensive similarity corresponding to each to-be-matched region, a target matching region is determined from each to-be-matched region.

2. The method according to claim 1, characterized in that Before determining a target matching area from each to-be-matched area according to the comprehensive similarity corresponding to each to-be-matched area, the method further includes: The next target template feature point of the template image is obtained according to the preset rule, and the process returns to the step of obtaining feature information of the target template feature point of the template image until all template feature points of the template image are obtained.

3. The method according to claim 2, characterized in that After all template feature points of the template image are acquired, the method further includes: moving the template image by a preset number of rows in a first moving direction or by a preset number of columns in a second moving direction, and returning to the step of calculating, in parallel, the similarities between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point based on the feature information, until the template image moves to coincide with a first boundary or a second boundary of the image to be matched; The first boundary is a boundary in the boundary of the image to be matched that corresponds to the first moving direction, and the second boundary is a boundary in the boundary of the image to be matched that corresponds to the second moving direction.

4. The method according to any one of claims 1 to 3, characterized in that Before calculating, for each target template feature point, the similarity between the target template feature point and a plurality of target image points corresponding to the target template feature point in the image to be matched based on the feature information of the target template feature point, the method further includes: For each target template feature point, based on single instruction stream multiple data stream SIMD, image information of multiple target image points corresponding to the target template feature point in the image to be matched is loaded from a cache; The method of calculating, for each target template feature point, the similarity between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point in parallel based on feature information of the target template feature point, includes: For each target template feature point, based on the image information and the feature information of the target template feature point, the similarity between each target image point and the target template feature point is calculated in parallel.

5. The method according to any one of claims 1 to 3, characterized in that The storage order of the feature information of each template feature point in the template image in the memory is consistent with the distribution order of the row and column where each template feature point is located in the template image; The acquiring feature information of the target template feature points of the template image includes: The feature information of the target template feature point is obtained according to the storage order of the feature information of each template feature point of the template image in the memory.

6. The method according to any one of claims 1 to 3, characterized in that The feature information of each template feature point includes the template offset value and image feature information of the template feature point; The method of calculating, for each target template feature point, the similarity between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point in parallel based on feature information of the target template feature point, includes: For each target template feature point, determine a plurality of target image points corresponding to the target template feature point in the image to be matched based on the template offset value of the target template feature point, position information of the image point corresponding to the reference point of the template image in the image to be matched, the column width of the image to be matched, and the preset number of columns or rows; The similarity between each target image point and the target template feature point is calculated in parallel based on the image feature information of the target template feature point and the image information of the plurality of target image points.

7. The method according to claim 6, characterized in that The determining, based on the template offset value of the target template feature point, position information of the image point corresponding to the reference point of the template image in the image to be matched, the column width of the image to be matched, and the preset number of columns or rows, of the target image points corresponding to the target template feature point in the image to be matched includes: The target image point currently corresponding to the target template feature point in the image to be matched is determined using the following formula based on the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched: p=i*w+j+offset Wherein, p is the target image point currently corresponding to the target template feature point in the image to be matched, i is the ordinal number of the row in which the image point corresponding to the reference point of the template image is located in the image to be matched, w is the column width or row width of the image to be matched, j is the ordinal number of the column in which the image point corresponding to the reference point of the template image is located in the image to be matched, and offset is the template offset value of the target template feature point, specifically expressed as vertical offset * w + horizontal offset; The template image is moved a preset number of columns in a third moving direction or a preset number of rows in a fourth moving direction, and the template offset value of the target template feature point, the position information of the image point corresponding to the reference point of the template image in the image to be matched, and the column width or row width of the image to be matched are returned, and the step of determining the target image point currently corresponding to the target template feature point in the image to be matched using the following formula is performed until the template image moves to coincide with the third boundary or the fourth boundary of the image to be matched, wherein the third boundary is the boundary in the boundary of the image to be matched corresponding to the third moving direction, and the fourth boundary is the boundary in the boundary of the image to be matched corresponding to the fourth moving direction.

8. An image template matching device, characterized in that: include: An image acquisition module is used to acquire a template image and an image to be matched; A feature information acquisition module is used to acquire feature information of a target template feature point of the template image, wherein the target template feature point is a template feature point of the template image acquired according to a preset rule; a similarity calculation module, configured to calculate, for each target template feature point, based on feature information of the target template feature point, the similarity between a plurality of target image points corresponding to the target template feature point in the image to be matched and the target template feature point in parallel, wherein the plurality of target image points are image points in the same row or column in the image to be matched, and two adjacent target image points are separated by a preset number of columns or rows; a similarity accumulation module, configured to accumulate the similarity between each target image point and the target template feature point to the comprehensive similarity corresponding to the to-be-matched region where the target image point is located, wherein the to-be-matched region where each target image point is located is: the region in the to-be-matched image corresponding to the template image when the target image point coincides with the target template feature point; The target region determination module is used to determine a target matching region from each to-be-matched region according to the comprehensive similarity corresponding to each to-be-matched region.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 7 are implemented.

Images