Target detection method and device based on template matching and storage medium

Abstract

The invention discloses a target detection method and device based on template matching and a storage medium. The method comprises the steps of obtaining an initial pyramid layer number corresponding to a template image; in response to the fact that mask information of a mask image corresponding to the template image does not meet a preset condition, the number of layers of the initial pyramid is adjusted according to the mask information, and the number of layers of a target pyramid is obtained; constructing a template image pyramid of the template image according to the number of layers of the target pyramid, and constructing a target image pyramid of a to-be-matched target image according to the number of layers of the target pyramid; and matching the top-layer template image in the template image pyramid with the top-layer target image in the target image pyramid to obtain a target object in the top-layer target image. According to the scheme, the target detection precision and detection efficiency based on template matching can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and particularly to an object detection method, device, and storage medium based on template matching. Background Art

[0002] Template matching is an image matching method widely used in the fields of image processing and computer vision, mainly used to detect whether there are parts in the target image that are similar or identical to the template image. Its core is to find the best matching position in the two images by comparing local regions in the template image and the target image.

[0003] Currently, the normalized cross-correlation algorithm (NCC) can be used for similarity measurement during template matching. It can effectively handle influencing factors such as illumination changes and contrast differences in images, improving the accuracy and robustness of template matching.

[0004] However, when using this algorithm for image matching analysis, the amount of computation is large and the calculation speed is slow, which is not an efficient choice. Summary of the Invention

[0005] The present application provides at least an object detection method, device, equipment, and computer-readable storage medium based on template matching.

[0006] In the first aspect of the present application, an object detection method based on template matching is provided, including:

[0007] Obtaining the initial pyramid level corresponding to the template image; in response to the mask information of the mask image corresponding to the template image not satisfying a preset condition, adjusting the initial pyramid level according to the mask information to obtain the target pyramid level; constructing a template image pyramid of the template image according to the target pyramid level, and constructing a target image pyramid of the target image to be matched according to the target pyramid level; matching the top template image in the template image pyramid with the top target image in the target image pyramid to obtain the target object in the top target image.

[0008] In an embodiment, the obtaining the initial pyramid level corresponding to the template image includes: obtaining the template size of the template image; performing successive downsampling processing on the template image according to the template size to obtain the sampled template image; in response to the current size of the sampled template image being smaller than the size threshold, determining the initial pyramid level according to the number of sampling times.

[0009] In one embodiment, adjusting the initial number of pyramid levels according to the mask information to obtain the target number of pyramid levels includes: obtaining the number of valid pixels in the mask information; and in response to the number of valid pixels being less than a pixel number threshold, performing a downscaling process on the initial number of pyramid levels to obtain the target number of pyramid levels.

[0010] In one embodiment, adjusting the initial number of pyramid levels according to the mask information to obtain the target number of pyramid levels includes: adjusting the initial number of pyramid levels until the mask information meets the preset condition to obtain a pending number of pyramid levels; determining the target information entropy corresponding to the template image and the mask image according to the pending number of pyramid levels; and in response to the target information entropy being less than an information entropy threshold, performing a downscaling process on the pending number of pyramid levels to obtain the target number of pyramid levels.

[0011] In one embodiment, determining the target information entropy corresponding to the template image and the mask image according to the pending number of pyramid levels includes: determining a pending top-level template image corresponding to the template image and a pending top-level mask image corresponding to the mask image according to the pending number of pyramid levels; determining the gray entropy of the pending top-level template image according to the valid pixels represented by the pending top-level mask image and the gray information of the pending top-level template image; calculating the gradient magnitude entropy and the gradient direction entropy of the pending top-level template image; and determining the target information entropy according to the gray entropy, the gradient magnitude entropy, and the gradient direction entropy.

[0012] In one embodiment, matching the top-level template image in the template image pyramid with the top-level target image in the target image pyramid to obtain the target object in the top-level target image includes: sliding and matching the top-level template image in the top-level target image according to a preset sliding step length, and calculating the target similarity score between the top-level target image and the top-level template image at each sliding position corresponding to the preset sliding step length; and determining the target object from the top-level target image according to the target similarity score.

[0013] In one embodiment, the top-layer template image includes multiple rotated template images, which are obtained by image rotation processing between different rotated template images. Sliding and matching the top-layer template image in the top-layer target image according to a preset sliding step length, and calculating the target similarity score between the top-layer target image and the top-layer template image at each sliding position corresponding to the preset sliding step length, includes: sliding and matching the top-layer target image with each rotated template image respectively to obtain a corresponding plurality of similarity score maps; performing a merging process according to the maximum similarity scores in each of the similarity score maps to obtain a target similarity score map, where the target similarity score map includes the target similarity score.

[0014] In one embodiment, determining a target object from the top-layer target image according to the target similarity score includes: determining the pixel points with the target similarity score greater than the similarity threshold as the relevant coordinate points of the target object; determining the position information of the target object according to the position information of the relevant coordinate points.

[0015] A second aspect of the present application provides a target detection device based on template matching, including: an acquisition module, configured to acquire an initial pyramid layer number corresponding to a template image; an adjustment module, configured to, in response to the mask information of the mask image corresponding to the template image not satisfying a preset condition, adjust the initial pyramid layer number according to the mask information to obtain a target pyramid layer number; a pyramid construction module, configured to construct a template image pyramid of the template image according to the target pyramid layer number, and construct a target image pyramid of the target image to be matched according to the target pyramid layer number; a matching module, configured to match a top-layer template image in the template image pyramid with a top-layer target image in the target image pyramid to obtain a target object in the top-layer target image.

[0016] A third aspect of the present application provides an electronic device, including a memory and a processor, where the processor is configured to execute program instructions stored in the memory to implement the above-mentioned target detection method based on template matching.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the above-mentioned target detection method based on template matching is implemented.

[0018] In the above solution, the initial number of pyramid levels corresponding to the template image is obtained; by comparing the mask information of the mask image corresponding to the template image with a preset condition, when the mask information does not meet the preset condition, the initial number of pyramid levels can be adjusted according to the mask information to obtain an adaptive target number of pyramid levels; a template image pyramid of the template image is constructed according to the target number of pyramid levels, and a target image pyramid of the target image to be matched is constructed according to the target number of pyramid levels; the top template image in the template image pyramid is matched with the top target image in the target image pyramid to obtain the target object in the top target image, thereby reducing the computational complexity in the image matching process, improving the efficiency of image matching, and also ensuring the target detection accuracy based on the adaptively adjusted number of pyramid levels.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit this application. Brief Description of the Drawings

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with this application and, together with the specification, are used to explain the technical solutions of this application.

[0021] Figure 1 is a schematic flowchart of an exemplary embodiment of the target detection method based on template matching of this application;

[0022] Figure 2 is an exemplary template image in the target detection method based on template matching of this application;

[0023] Figure 3 is an exemplary mask image in the target detection method based on template matching of this application;

[0024] Figure 4 is an exemplary top template image in the target detection method based on template matching of this application;

[0025] Figure 5 is an exemplary top template gradient magnitude map in the target detection method based on template matching of this application;

[0026] Figure 6 is an exemplary top template gradient direction map in the target detection method based on template matching of this application;

[0027] Figure 7 is an exemplary scene diagram for constructing an image pyramid in the target detection method based on template matching of this application;

[0028] Figure 8 is an exemplary scene diagram for image rotation processing in the target detection method based on template matching of this application;

[0029] Figure 9 is an exemplary target image in the template matching-based target detection method of the present application;

[0030] Figure 10 is an exemplary target image pyramid in the template matching-based target detection method of the present application;

[0031] Figure 11 Result graph of the top-level normalized cross-correlation metric in the template matching-based target detection method of the present application;

[0032] Figure 12 is the angle graph corresponding to the maximum value of the top-level normalized cross-correlation metric in the template matching-based target detection method of the present application;

[0033] Figure 13 is a schematic diagram of pyramid iterative matching in the template matching-based target detection method of the present application;

[0034] Figure 14 is a schematic diagram of the matching result in the template matching-based target detection method of the present application;

[0035] Figure 15 is a block diagram of the template matching-based target detection device shown in an exemplary embodiment of the present application;

[0036] Figure 16 is a schematic structural diagram of an embodiment of an electronic device of the present application;

[0037] Figure 17 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. Detailed implementation manners

[0038] The solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings of the specification.

[0039] In the following description, specific details such as specific system architectures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, in order to thoroughly understand the present application.

[0040] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. In addition, "multiple" in this article means two or more than two. In addition, the term "at least one" in this article represents any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0041] Template matching is a recognition method widely used in the fields of image processing and computer vision, mainly used to detect whether there are parts in the target image that are similar to the template image. The core of this technology is usually to compare some or all regions in the template image and the target image to find the best matching position between the two. Typical application scenarios of template matching include but are not limited to object detection and recognition, image stitching and registration, character recognition, video analysis, etc. In this application, object detection is mainly used as an example for illustration, but according to different application scenarios, the method of this application can also be applied to scenarios such as image stitching and registration, character recognition, video analysis, etc. For example, the target object of this application can be animals and plants, physical objects, characters, and / or some texture features, etc., which are not limited here.

[0042] It should also be noted that Normalized Cross-Correlation (NCC) is one of the most commonly used similarity metrics in template matching. It can effectively handle influencing factors such as illumination changes and contrast differences, and improve the accuracy and robustness of matching.

[0043] Please refer to Figure 1 , Figure 1 is a schematic flowchart of an exemplary embodiment of the object detection method based on template matching of this application. Specifically, it can include the following steps:

[0044] Step S110, obtain the initial pyramid level corresponding to the template image.

[0045] The template image is an image used to provide reference information in the image matching process. For example, the target object may exist in the template image. By matching the template image and the target image, it is possible to determine whether the target object exists in the target image, as well as relevant information such as the position of the target object in the target image.

[0046] The initial pyramid level can be used to construct an image pyramid. The image pyramid can be used to reduce the computational amount in image matching and achieve the beneficial effect of accelerating calculation. However, in the process of constructing the image pyramid of the current mainstream algorithms, the pyramid level needs to be determined by empirical values obtained through multiple experiments, and it is difficult to balance issues such as matching accuracy and matching efficiency.

[0047] Exemplarily, in practical application scenarios, there may be one or more template images, and each template image may be preset with its corresponding initial pyramid level according to empirical values. When performing template matching, the corresponding initial pyramid level can be obtained according to the currently used template image for image pyramid construction.

[0048] Optionally, in the present application, the initial number of pyramid levels corresponding to the template image can be not only preset according to empirical values, but also dynamically determined according to the size information of the template image. It can be understood that during the process of constructing the image pyramid, when the image is raised by one level (equivalent to performing one downsampling), its size information will decrease. Therefore, the present application can preset a size threshold corresponding to the template image, and constrain the number of samplings of the template image through the size threshold, so as to avoid the situation where the initial number of pyramid levels is set too high according to empirical values, resulting in the failure to successfully match the target object, and thus obtain the initial number of pyramid levels corresponding to the template image.

[0049] In summary, in the present application, the initial number of pyramid levels corresponding to the template image can be preset according to empirical values, and / or determined according to the template size information of the template image, and no limitation is made here.

[0050] Step S120, in response to the mask information of the mask image corresponding to the template image not meeting the preset conditions, adjust the initial number of pyramid levels according to the mask information to obtain the target number of pyramid levels.

[0051] The mask image is an image used to perform masking processing on the template image. The mask image is usually a two-dimensional array with the same size as the template image, and each pixel value on the mask image can be used to indicate the processing method corresponding to the pixel in the template image. The mask image mainly uses two basic values: zero value (usually black, 0) and non-zero value (usually white, 255). Among them, the zero-value pixels can be called invalid pixels, and the image area represented by the zero-value pixels is also often called the invalid area; the non-zero-value pixels can be called valid pixels, and the image area represented by the non-zero-value pixels can also be called the valid area. It can be understood that these values also represent the region of interest (ROI) and the non-region of interest respectively, and no further elaboration is made here.

[0052] Among them, the mask information of the mask image includes the number of valid pixels (the number of non-zero pixel points) and / or the number of invalid pixels (the number of zero pixel points) of the mask image, etc.

[0053] It should be noted that in the conventional template matching technology, the template image can be directly matched with the target image to determine the image area in the target image that is similar or identical to the template image. However, such methods are easily affected by the non-region of interest in the template image, thereby reducing the accuracy of image matching.

[0054] Therefore, in the present application, it is possible to directly perform matching processing between the template image and the target image by using the conventional template matching technology, or it is possible to perform masking processing on the template image by using the mask image and then match the masked template image with the target image, thereby improving the accuracy of image matching.

[0055] Combined with the foregoing description, in order to improve the accuracy of image matching, the template image participating in image matching can be masked according to the mask image; and in order to improve the efficiency of image matching, in this application, the template image will be downsampled (constructing a template image pyramid to obtain the top-layer template image), and the template size of the template image will gradually decrease accordingly. Therefore, the mask image also needs to be processed in the same way with reference to the construction process of the template image pyramid to obtain the top-layer mask image. Thus, the image sizes of the top-layer mask image and the top-layer template image match each other, that is, the mask image at the same level in the image pyramid can be used to mask the template image (for example, using the top-layer mask image to mask the top-layer template image).

[0056] Therefore, the preset conditions of this application are set for the mask information, which can be analyzed from the perspective of the mask image. By comparing the mask information with the preset conditions, it can be judged whether the currently determined initial pyramid level meets the requirements, and thus it can be decided whether to adjust the initial pyramid level. If the initial pyramid level is not adjusted, the initial pyramid level can be determined as the target pyramid level; if the initial pyramid level is adjusted, the adjusted initial pyramid level can be determined as the target pyramid level.

[0057] Step S130, construct a template image pyramid of the template image according to the target pyramid level, and construct a target image pyramid of the target image to be matched according to the target pyramid level.

[0058] Among them, the target image refers to the image for which image matching is required. The application scenario of the template matching technology is usually to match the template image with the target image to find similar or identical image regions in the two images, which will not be elaborated here.

[0059] Combined with the foregoing steps for description, after obtaining the target pyramid level, a template image pyramid of the template image and a target image pyramid of the target image can be constructed respectively. There can be one or more methods for constructing an image pyramid, and the specific process can refer to the existing image pyramid construction methods, which will not be elaborated here.

[0060] It should be noted that in the actual application process, the pyramid levels of the template image pyramid and the target image pyramid may be the same as or different from the target pyramid level. Ideally, when the pyramid levels of the template image pyramid and the target image pyramid are the same as the target pyramid level, the image matching accuracy and the image matching efficiency can be taken into account to achieve the optimal image matching effect.

[0061] Step S140: Match the top - layer template image in the template image pyramid with the top - layer target image in the target image pyramid to obtain the target object in the top - layer target image.

[0062] It can be understood that there are multiple levels of images in the image pyramid, which will not be elaborated here. The top - layer image in the image pyramid refers to the image at the highest level in the image pyramid (for example, if the pyramid levels of the image pyramid are numbered from 1 to N from bottom to top, then the N - th layer image is the top - layer image), and it is usually the image with the smallest size or resolution.

[0063] Therefore, after obtaining the template image pyramid and the target image pyramid, the top - layer template image in the template image pyramid and the top - layer target image in the target image pyramid can be obtained. By matching the top - layer template image with the top - layer target image, compared with directly matching the template image with the target image in the traditional method, the computational amount of the matching calculation is significantly reduced, and the matching efficiency is improved.

[0064] Among them, the image matching method of this application is a template - matching method based on an image template, which can also be called a sliding - matching method. It mainly compares the template image with an image region of the same size on the target image through a sliding window, and searches for similar or identical image regions from it, thereby obtaining the target object or target region. For example, sliding the template image on the target image (or sliding the top - layer template image on the top - layer target image), and calculating the similarity between the template image and the corresponding region on the target image, so as to determine the best matching position between the two images. The specific details will not be elaborated here.

[0065] It can be seen that this application obtains the initial pyramid level corresponding to the template image; compares the mask information of the mask image corresponding to the template image with the preset conditions, and when the mask information does not meet the preset conditions, the initial pyramid level can be adjusted through the mask information to obtain the adaptive target pyramid level; constructs the template image pyramid of the template image according to the target pyramid level, and constructs the target image pyramid of the target image to be matched according to the target pyramid level; matches the top - layer template image in the template image pyramid with the top - layer target image in the target image pyramid to obtain the target object in the top - layer target image. Thus, the computational amount in the image - matching process can be reduced, the efficiency of image matching can be improved, and the target - detection accuracy can also be guaranteed based on the adaptively adjusted pyramid level.

[0066] Based on the above - mentioned embodiments, the embodiment of this application describes the step of obtaining the initial pyramid level corresponding to the template image. Specifically, the method of this embodiment includes the following steps:

[0067] Obtain the template size of the template image; perform successive downsampling processing on the template image according to the template size to obtain the downsampled template image; in response to the current size of the downsampled template image being smaller than the size threshold, determine the initial pyramid layer number according to the number of sampling times.

[0068] Combined with the foregoing embodiments for illustration, the pyramid layer number set according to empirical values is difficult to adapt to different application scenarios. If the pyramid layer number is too high, it may lead to failure to match the target. If the pyramid layer number is too low, it may lead to too long matching time.

[0069] Therefore, in the specific implementation process of this application, it is possible to select to determine the appropriate initial pyramid layer number according to the template size of the template image. Since the size (resolution) of the image decreases as each layer of the image pyramid is raised, this application can use a preset size threshold (or resolution threshold) to constrain the initial pyramid layer number to prevent too many feature information losses in the top-layer image due to too high pyramid layer number.

[0070] Taking the template size as an example for illustration, obtain the template image I tpl The template size of includes width W tpl and height H tpl . In the process of building the image pyramid, when processing according to the preset downsampling step, the image size of each corresponding level of the image pyramid will decrease as the pyramid is raised one layer. For example, when building the image pyramid with a downsampling step of 2, the image size will be halved as the image pyramid is raised one layer. It is equivalent to in the image pyramid, the size of the Nth layer is half of the (N - 1)th layer, and the mathematical expressions for its width and height can be respectively:

[0071]

[0072] Correspondingly, the preset size threshold can be denoted as width threshold W th and height threshold H th , where the specific values of the width threshold and the height threshold can be the same or different, which is not limited here. The constraint provided by the size threshold can prevent the current size of the downsampled template image from being too small or too large.

[0073] For example, W th = 12, H th = 12. According to the construction principle of the image pyramid, calculate the current size of the downsampled template image in each level until the current width in the current size < 12 and / or the current height < 12, then the corresponding initial pyramid layer number pyr can be determined according to the current number of sampling times (or according to the layer number where the current top-layer template image is located). That is, the condition that the initial pyramid layer number pyr should satisfy is: the width W of the template image (top-layer template image) of the pyrth layer pyr < Wth and / or height H pyr < H th If pyr is too low, the template size of the template image at the pyr layer will be greater than or equal to the size threshold. Therefore, downsampling can be continued to increase the number of layers of the image pyramid until the current size of the template image at the pyr layer is less than the size threshold. Then, the initial pyramid layer number can be determined according to the number of sampling times (or the layer number where the current top template image is located). Specifically, the number of sampling times can be determined as the initial pyramid layer number, or the initial pyramid layer number can be obtained after appropriately adjusting the number of sampling times, which is not limited here. For example, (the number of sampling times - 1) can be determined as the initial pyramid layer number, or (the number of sampling times + 1) can be determined as the initial pyramid layer number, etc.

[0074] Optionally, in addition to determining the corresponding initial pyramid layer number according to the width and / or height of the template image, it can also be determined in the same way according to the image area of the template image, etc. (for example, comparing the current image area of the sampled template image with the image area threshold to determine whether to increase the number of pyramid layers), which is not limited here.

[0075] Based on the above embodiments, the embodiments of the present application will describe the steps of adjusting the initial pyramid layer number according to the mask information to obtain the target pyramid layer number. Specifically, the method of this embodiment includes the following steps:

[0076] Obtain the number of valid pixels in the mask information; in response to the number of valid pixels being less than the pixel number threshold, perform a reduction process on the initial pyramid layer number to obtain the target pyramid layer number.

[0077] Combined with the foregoing embodiments, in different application scenarios, if the size of the region of interest is large, the low-resolution image or rough view after multiple downsamplings can also provide rich features. For a region of interest with a small size, it is necessary to ensure that the image features are fully retained in high-resolution form for image matching to avoid affecting the image matching accuracy. Therefore, the initial pyramid layer number obtained according to the template image in the method of the foregoing embodiments may not necessarily be the most suitable for the current application scenario.

[0078] In this embodiment, the number of valid pixels in the mask information can be analyzed, and whether to further adjust the initial pyramid layer number can be determined based on the comparison result between the number of valid pixels and the preset pixel number threshold.

[0079] Exemplarily, denote the number of valid pixels (non-zero pixel points) in the mask image I msk as cnt notzero . The image size of the mask image I msk and the template image I tplThe image sizes are the same, so that the region of interest in the target image can be accurately determined through the mask image. It can be understood from the foregoing embodiments that for the mask image, when the number of pyramid levels increases step by step, the number of effective pixels cnt for each layer corresponding to the mask image notzero will also become smaller accordingly. Therefore, when masking the top-level template image according to the top-level mask image, if the number of effective pixels in the top-level mask image is too small, it will affect the analysis of the region of interest in the top-level template image and reduce the image matching accuracy.

[0080] On the one hand, after determining the initial pyramid level pyr, a mask image pyramid corresponding to the mask image can be constructed according to the initial pyramid level, and the mask image at the pyr-th level (top-level mask image) can be determined, and the number of effective pixels of the top-level mask image is obtained and compared with the first pixel number threshold. If the number of effective pixels is less than the first pixel number threshold, the initial pyramid level is adjusted downward, and the number of effective pixels of its top-level mask image is re-determined until the number of effective pixels is greater than or equal to the first pixel number threshold, and the target pyramid level is obtained. If the number of effective pixels is greater than or equal to the first pixel number threshold, the initial pyramid level can be directly determined as the target pyramid level.

[0081] Optionally, in addition to adjusting the initial pyramid level according to the comparison result between the number of effective pixels in the top-level mask image and the pixel number threshold in the foregoing embodiments, it is also possible to decide whether to adjust the initial pyramid level according to the comparison result between the number of effective pixels in the original mask image and the second pixel number threshold. The first pixel number threshold and the second pixel number threshold can be the same or different, which is not limited here. In this embodiment, if the number of effective pixels in the original mask image is small (less than the second pixel number threshold), the number of effective pixels in the top-level mask image obtained after its downsampling will also be insufficient, and the initial pyramid level needs to be adjusted downward to obtain the target pyramid level. Among them, in this embodiment, the initial pyramid level can be adjusted downward according to the preset adjustment step step (for example, if the number of effective pixels in the original mask image is less than the second pixel number threshold, the initial pyramid level is subtracted by step); vice versa, which is not elaborated here.

[0082] On the other hand, there is another implementable manner: obtaining the number of invalid pixels in the top-layer mask image and comparing it with a third pixel number threshold. The value of the third pixel number threshold may be the same as or different from the first pixel number threshold and the first pixel number threshold in the previous embodiments, which is not limited herein. If the number of invalid pixels is greater than the third pixel number threshold, the initial number of pyramid levels is adjusted downward, and the number of invalid pixels in its top-layer mask image is re-determined until the number of invalid pixels is less than or equal to the third pixel number threshold, obtaining the target number of pyramid levels. If the number of invalid pixels is less than or equal to the third pixel number threshold, the initial number of pyramid levels may be directly determined as the target number of pyramid levels.

[0083] Based on the above embodiments, the embodiments of the present application illustrate the steps of adjusting the initial number of pyramid levels according to mask information to obtain the target number of pyramid levels. Specifically, the method of this embodiment includes the following steps:

[0084] Adjust the initial number of pyramid levels until the mask information meets the preset conditions to obtain a pending number of pyramid levels; determine the target information entropy corresponding to the template image and the mask image according to the pending number of pyramid levels; in response to the target information entropy being less than the information entropy threshold, adjust the pending number of pyramid levels downward to obtain the target number of pyramid levels.

[0085] Combined with the foregoing embodiments for explanation, in the present application, in addition to adjusting the initial number of pyramid levels according to mask information such as the number of valid pixels in the mask image, it is also possible to adjust according to the target information entropy corresponding to the template image and the mask image.

[0086] Exemplarily, according to the method of the foregoing embodiments, adjust the initial number of pyramid levels until the mask information meets the preset conditions (for example, the number of valid pixels in the top-layer mask image corresponding to the adjusted number of pyramid levels is greater than or equal to the pixel number threshold, etc.), then the adjusted number of pyramid levels can be determined as the pending number of pyramid levels. Then, through the comparison result between the target information entropy and the information entropy threshold, it is decided whether to adjust the pending number of pyramid levels to obtain the target number of pyramid levels, so as to further improve the matching accuracy.

[0087] Among them, if the target information entropy is less than the information entropy threshold, the pending number of pyramid levels is adjusted downward to obtain the target number of pyramid levels; if the target information entropy is greater than or equal to the information entropy threshold, the pending number of pyramid levels can be determined as the target number of pyramid levels.

[0088] Based on the above embodiments, the embodiments of the present application illustrate the steps of determining the target information entropy corresponding to the template image and the mask image according to the pending number of pyramid levels. Specifically, the method of this embodiment includes the following steps:

[0089] Determine the to-be-determined top-layer template image corresponding to the template image and the to-be-determined top-layer mask image corresponding to the mask image according to the to-be-determined number of pyramid layers; determine the gray entropy of the to-be-determined top-layer template image according to the effective pixels represented by the to-be-determined top-layer mask image and the gray information of the to-be-determined top-layer template image; perform gradient calculation on the to-be-determined top-layer template image to obtain the gradient magnitude entropy and gradient direction entropy of the to-be-determined top-layer template image; determine the target information entropy according to the gray entropy, gradient magnitude entropy and gradient direction entropy.

[0090] Taking the foregoing embodiments as an example for illustration, after obtaining the to-be-determined number of pyramid layers, the to-be-determined top-layer template image I corresponding to the template image can be determined according to the to-be-determined number of pyramid layers tpl_pyr and the to-be-determined top-layer mask image I corresponding to the mask image msk_pyr . The specific method can still refer to the downsampling process and image pyramid construction process described in the foregoing embodiments, etc., and will not be elaborated here.

[0091] Then, according to the gray information of the to-be-determined top-layer template image and the effective pixels represented by the to-be-determined top-layer mask image, the gray histogram distribution P of the to-be-determined top-layer template image can be calculated i , P i represents the probability that the gray value i appears in the to-be-determined top-layer template image I tpl_pyr . That is, when calculating the P of the to-be-determined top-layer template image i , the to-be-determined top-layer template image is masked by the to-be-determined top-layer mask image, so that the calculation process only considers the positions in the to-be-determined top-layer template image corresponding to the effective pixels in the to-be-determined top-layer mask image I msk_pyr (equivalent to the positions corresponding to non-zero pixels or the positions corresponding to pixel values of 1, etc.). Thus, the gray one-dimensional entropy H1 of the to-be-determined top-layer template image can be calculated, and its mathematical expression can be:

[0092]

[0093] Furthermore, perform gradient calculation on the to-be-determined top-layer template image to obtain the gradient magnitude entropy and gradient direction entropy of the to-be-determined top-layer template image. Specifically, the sobel operator can be used to calculate the horizontal gradient image G tpl_pyr and the vertical gradient image G x of the to-be-determined top-layer template image I y . The specific calculation principle can refer to the existing gradient calculation methods, which will not be elaborated here, and its mathematical expression can be:

[0094]

[0095] The mathematical expression of the gradient magnitude map I amp_pyr can be:

[0096] I amp_pyr= |G x | + |G y |

[0097] Gradient direction map I dir_pyr can be mathematically expressed as:

[0098]

[0099] Then, by the same token, referring to the method of calculating the gray entropy H1, replace the variable P i correspondingly, and respectively calculate the gradient magnitude entropy H2 and the gradient direction entropy H3 according to the gradient magnitude map I amp_pyr and the gradient direction map I dir_pyr Thus, the target information entropy H can be determined by the gray entropy H1, the gradient magnitude entropy H2, and the gradient direction entropy H3, and its mathematical expression can be:

[0100] H = H1 + H2 + H3

[0101] Combined with the foregoing embodiments for illustration, subsequently, the target information entropy H can be compared with the information entropy threshold H th If H < H th , then the method of the foregoing embodiments can be referred to for reducing the number of pending pyramid levels. For example, pyr = pyr - 1 can be referred to for iterative reduction, reducing one level each time until the target information entropy H is greater than or equal to the information entropy threshold H th . If H ≥ H th , then the number of pending pyramid levels can be determined as the target pyramid level.

[0102] In summary, in the specific implementation process of this application, the initial pyramid level determined by the template size can be determined as the target pyramid level for constructing the image pyramid, or the target pyramid level obtained by adjusting the initial pyramid level according to the mask information of the mask image can be used for constructing the image pyramid, or the target pyramid level obtained by adjusting the number of pending pyramid levels according to the target information entropy can be used for constructing the image pyramid, etc., which is not limited here.

[0103] Based on the above embodiments, the embodiments of this application illustrate the steps of matching the top - layer template image in the matching template image pyramid with the top - layer target image in the target image pyramid to obtain the target object in the top - layer target image. Specifically, the method of this embodiment includes the following steps:

[0104] Slide - match the top - layer template image in the top - layer target image according to a preset sliding step size, and calculate the target similarity score between the top - layer target image and the top - layer template image at each sliding position corresponding to the preset sliding step size; determine the target object from the top - layer target image according to the target similarity score.

[0105] Illustrated with reference to the foregoing embodiments, in the process of image matching, the existing template-based image matching method can be similarly referred to. For example, the template image is slid on the target image for matching. In the specific implementation process of this application, it is possible to choose to slide and match the top-level template image on the top-level target image, thereby significantly reducing the computational amount.

[0106] Among them, the preset sliding step refers to the step length by which the image moves each time during the sliding matching process. Exemplarily, the sliding step can be set to 1 pixel point, which means that the top-level template image can move 1 pixel point (usually a parallel movement) each time an image matching is completed on the top-level target image, until all pixel points in the top-level target image are image-matched with the top-level template image, or until the target object in the top-level target image is matched. This application does not limit the conditions for suspending or stopping image matching, and can be correspondingly set according to the specific implementation scenario.

[0107] Exemplarily, when performing image matching processing between the top-level target image and the top-level template image at each sliding position, the similarity between each pixel point of the top-level target image and the top-level template image is calculated in the image area corresponding to this position, and the target similarity score is obtained. After the sliding matching is completed, the target similarity score corresponding to each pixel point in the top-level target image can be obtained. By comparing the target similarity score corresponding to each pixel point with the preset similarity threshold, the pixel points with the target similarity score greater than the similarity threshold can be determined as the relevant pixel points of the target object. Based on these relevant pixel points, the image area where the target object is located can be determined, and the target object and the relevant information of the target object (such as position information, edge information, size information, etc. The specific calculation method can refer to the existing method and will not be elaborated here) in the top-level target image can be obtained.

[0108] Furthermore, the foregoing method can be used to detect whether there is a target object in the target image. However, if it is necessary to accurately determine the relevant information of the target object, since the top-level target image is obtained through downsampling processing, it is also necessary to map it to the original target image, so as to obtain the target object in the target image. The specific method can be to update the position information of the target object in each level image of the image pyramid sequentially from the top layer of the target image pyramid according to the downsampling step length when constructing the image pyramid and the position information of the target object determined in the top-level target image.

[0109] For example, if the coordinate information of the target object in the top-level target image (the pyr layer) is (x obj , y obj ), and the downsampling step length is 2, then the coordinate information of the target object in the image of the pyr - 1 layer of the target image pyramid is (x c , y c) = (2 * x obj , 2 * y obj )。

[0110] Based on the above embodiments, the embodiments of the present application will describe the steps of sliding and matching the top - layer template image in the top - layer target image according to a preset sliding step size, and calculating the target similarity score between the top - layer target image and the top - layer template image at each sliding position corresponding to the preset sliding step size. Among them, the top - layer template image includes multiple rotated template images, and different rotated template images are obtained through image rotation processing. Specifically, the method of this embodiment includes the following steps:

[0111] Perform sliding matching on the top - layer target image with each rotated template image respectively to obtain a corresponding plurality of similarity score maps; perform merging processing according to the maximum similarity scores in each similarity score map to obtain a target similarity score map, and the target similarity score map includes the target similarity score.

[0112] It should be noted that in the traditional template matching method, the template image matches the similarity between each pixel point by parallel movement in the target image. In some special application scenarios, the template matching has certain limitations due to its matching principle. For example, if there is a rotational change of the target object in the target image relative to the target object in the template image, it is difficult for the common template matching method to accurately determine the target object in the target image through template matching.

[0113] Therefore, in the specific implementation manner of the present application, the template image for the same target object may include one or more, and multiple template images can be obtained by rotating an original template image through image rotation processing. Taking the top - layer template image as an example, after determining the number of target pyramid layers in the foregoing embodiments, a template image pyramid can be constructed according to the number of target pyramid layers. Then, image rotation processing can be performed on the top - layer template image in the template image pyramid to obtain multiple top - layer template images (i.e., rotated template images).

[0114] Exemplarily, assume that the angle range to be matched for the target object to be matched is [minAngle, maxAngle]. First, it is necessary to calculate the angle step size for top - layer search at the top layer of the image pyramid. In combination with the foregoing embodiments, if the width and height of the template image I tpl are W tpl and H tpl respectively, then half of the diagonal length of the template image is radius, and its mathematical expression is:

[0115]

[0116] Therefore, the mathematical expression of the angle step angleStep during the rotation process of the top-layer template image can be:

[0117]

[0118] It can be seen therefrom that its geometric meaning is that when the top-layer template image rotates by angleStep degrees, the endpoints of the image diagonal will move 1 pixel. In summary, when constructing the image pyramid, if the downsampling step is "2", the angle step angleStep of the k-th layer in the image pyramid k The mathematical expression can be:

[0119] angleStep k = angleStep * 2 k-1

[0120] That is to say, when the target pyramid layer number is pyr, the mathematical expression of the angle step (or the top-layer angle step) angleStepTop of the top-layer template image can be:

[0121] angleStepTop = angleStep * 2 pyr-1

[0122] Similarly, the angle steps of the top-layer mask image and the top-layer template image can be shared. That is, after obtaining the above angle step, the top-layer template image and the top-layer mask image can be rotated within the angle range [minAngle, maxAngle] according to the angle step angleStepTop, and the rotation angles can be minAngle, minAngle + angleStepTop, minAngle + 2 * angleStepTop,..., maxAngle. Thus, multiple rotated template images can be obtained.

[0123] Furthermore, establish the target image pyramid of the target image according to the target pyramid layer number. Assume that one of the rotated template images within the angle range [minAngle, maxAngle] is T and the top-layer target image is I. Then, the image T can be slid on the image I according to the sliding step. For example, the sliding step is 1 pixel point, and matching calculations are performed at each sliding position to obtain the corresponding similarity scores. Among them, the method for calculating the similarity scores can refer to the Normalized Cross-Correlation (NCC) algorithm, and the calculated normalized cross-correlation metric is used as the similarity score to form the metric score map S.

[0124] Among them, the mathematical expression of the common ncc algorithm is:

[0125]

[0126] t(u, v) is the gray value at position (u, v) in the template image, and i(r + u, c + v) is the gray value at position (u, v) in the target image, where m t is the average gray value of the template image, and m i (r, c) is the average gray value of the corresponding window at position (r, c) in the target image, and s t 2 is the gray variance of the template image, and s i 2 (r, c) is the gray variance of the corresponding window at position (r, c) in the target image. R is the effective area of the template image, that is, R is the image area with non - zero gray values in the mask image, and n is the total number of pixel points in the effective area of the template image. However, the computational complexity of the original NCC algorithm is very high. The simplified mathematical expression in this application is:

[0127]

[0128] where u t and u i are the average gray values of the template image and the target image respectively, and n is the total number of pixel points in the effective area of the template image. Based on the foregoing embodiments, after rotating the top - layer template image and the mask image within the angle range [minAngle, maxAngle], the total number of effective pixels n, the average gray value u t of the template image, and the cumulative value ∑t 2 (u, v) of the squared gray values of the template image can be calculated respectively.

[0129] Since each rotated template image can be slid - matched with the top - layer target image, after the sliding match is completed, each rotated template image can have a corresponding metric score map.

[0130] After traversing all the top - layer template images within the angle range [minAngle, maxAngle] according to the above method and calculating the corresponding metric score maps respectively, the rotation angles corresponding to each metric score map can be recorded simultaneously. If there are m rotated template images, then there can be m metric score maps S1, S2, …, S m , and the corresponding angles are angle1, angle2, …, angle m . Then, the m metric score maps can be merged into one score map to obtain the target similarity score map S max , and the elements of S max are the maximum values of the corresponding positions of the m score maps, which is equivalent to S max (x, y) = max{S1(x, y), S2(x, y), …, S m(x, y)}, the target similarity score map includes the target similarity score. That is, for each pixel position, the maximum similarity score at that pixel position is selected from each similarity score map as the target similarity score at that pixel position in the target similarity score map. And an angle map A can also be calculated, where each element in map A is the angle corresponding to each element in score map S max in score map S

[0131] Based on the above embodiments, the embodiments of the present application illustrate the steps of determining the target object from the top-level target image according to the target similarity score. Specifically, the method of this embodiment includes the following steps:

[0132] Determine the relevant coordinate points of the target object for the pixel points where the target similarity score is greater than the similarity threshold; determine the position information of the target object according to the position information of the relevant coordinate points.

[0133] Combined with the foregoing embodiments for description, for the pixel points where the target similarity score is greater than the similarity threshold, it can be considered as the pixel points in the top-level target image that are similar or identical to the top-level template image, that is, the pixel points (relevant coordinate points) equivalent to the target object. Thus, the position information of the target object can be determined according to the position information of the relevant coordinate points. For example, the relevant coordinate points can represent the corner points of the circumscribed rectangle of the target object, and the centroid coordinates of the target object can be determined according to the corner point coordinates through the corresponding centroid conversion method, thereby obtaining the position information of the target object.

[0134] In addition, according to the actual requirements in the specific implementation process, other coordinate calculation methods can also be used to represent the position information of the target object, which will not be elaborated here.

[0135] And determining the position information of the target object according to the position information of the relevant coordinate points can be to determine the position information of the target object in the top-level target image, and / or to determine the position information of the target object in other level images in the target image pyramid, which is not limited here.

[0136] On the one hand, the target similarity score can be compared with the similarity threshold, and the position information and angle information of the target object can be determined according to the comparison result (for example, the pixel points where the target similarity score is greater than the similarity threshold are determined as the pixel points corresponding to the target object).

[0137] On the other hand, it can be to extract the local maximum points in the target similarity score map S max in score map S max in score map S. Assuming that point P(x, y) is a local maximum point, the conditions that point P needs to satisfy are: in S

[0138]

[0139] Thus, all local maximum points extracted from S max can be saved in the candidate target queue and arranged in descending order according to the target similarity scores. For each local maximum stored in the candidate target queue, it can be filtered through a preset maximum value threshold, and the local maximum points greater than the maximum value threshold are retained to obtain candidate matching targets. In this embodiment, the method of threshold screening can further improve the accuracy of target detection. Among them, combining the score map S max and the angle map A, the candidate matching targets in the top-level target image can be denoted as cand obj = {x obj , y obj , angle obj , score obj}, where x obj , y obj are the position coordinates of the candidate matching target in the top-level target image, angle obj is its corresponding rotation angle, and score obj is its corresponding target similarity score. The upper limit of the number of objs can be the number of all local maximum points with target similarity scores greater than the similarity threshold.

[0140] After obtaining the candidate matching targets in the top-level target image through the foregoing method, the similarity of the neighborhood of each candidate matching target in the candidate target queue can be calculated layer by layer downward along the target image pyramid, and the position of the candidate matching target in each layer image of the pyramid can be updated. For example, if one of the candidate matching targets in the top-level target image (the pyr layer) is cand obj = {x obj , y obj , angle obj , score obj}, then the state of this candidate matching target in the next layer (the pyr - 1 layer) of the top level needs to be updated. According to the method provided in the foregoing embodiment, the angle step corresponding to the pyr - 1 layer is angleStep pyr-1 = angleStep * 2 pyr-2 , and the 5 calculation angles selected in the pyr - 1 layer can be: angle obj - 2 * angleStep pyr-1 , angle obj - angleStep pyr-1 , angle obj , angle obj + angleSteppyr-1 , angle obj + 2 * angleStep pyr-1 .

[0141] Since cand obj is at position (x obj , y obj ) at the top layer, if the downsampling step is 2, then its position at the pyr - 1 layer is (x c , y c ) = (2 * x obj , 2 * y obj ). Selecting the position according to this position coordinate and calculating with a preset specification (such as 5×5), we can get:

[0142]

[0143] Calculate the similarity scores of 5 angles respectively at the 5×5 position of the pyr - 1 layer pyramid, select the maximum value and record its position, and update the candidate information of the candidate target at the pyr - 1 layer. For example, if the normalized cross - correlation metric is at (x c - 1, y c + 1) and the maximum value is score objnew , and the corresponding angle is angle obj + angleStep pyr-1 , then the updated position of the candidate matching target cand obj at the pyr - 1 layer is:

[0144] cand obj = {x c - 1, y c + 1, angle obj + angleStep pyr-1 , score objnew}

[0145] After completing the update of all candidate matching targets at the pyr - 1 layer as above, repeat the above process to complete the calculation of the normalized cross - correlation metrics of the candidate matching targets at the pyr - 2, pyr - 3,..., 1 layers respectively.

[0146] After calculating the normalized cross-correlation metric for each image in the target image pyramid, for each layer of images, the candidate matching targets in each layer of images can be sorted according to the normalized cross-correlation metric from largest to smallest, and the candidate matching targets that do not meet the standard of the normalized cross-correlation metric (target similarity score) can be deleted according to a preset similarity threshold. Since zero, one, or more target objects may be determined in the target image, for each layer of images, the IOU values between the candidate matching targets in each layer of images can also be calculated, and according to a preset overlap rate threshold, the candidate matching target with a smaller target similarity score among the two candidate matching targets with an IOU value greater than the overlap rate threshold can be deleted. In summary, the remaining candidate matching targets in each layer of images can be determined as target objects.

[0147] Further, for each target object obtained in the foregoing embodiment, its position coordinates and the normalized cross-correlation metric within the 3-neighborhood range of the angle are obtained. Among them, the 3-neighborhood range of the present application refers to x plus or minus 1, y plus or minus 1, and angle plus or minus angleStep. Therefore, there are a total of 3×3×3 = 27 normalized cross-correlation metrics within the 3-neighborhood range of the target object.

[0148] Perform surface fitting on the position, angle, and normalized cross-correlation metric of the target object. Let x, y, and θ represent the position coordinates and angle respectively, and f(x, y, θ) represent the normalized cross-correlation metric score. The mathematical expression of the fitting equation can be:

[0149] f(x, y, θ) = k0x 2 +k1y 2 +k2θ 2 +k3xy + k4xθ + k5yθ + k6x + k7y + k8θ + k9

[0150] Converting the above fitting equation into matrix form gives:

[0151]

[0152] Therefore, the values of x, y, and θ can be calculated by the following formula:

[0153]

[0154] Its final mathematical expression can be:

[0155]

[0156] Based on the implementation process of the foregoing embodiment, after performing surface fitting on each target object, the obtained values of x, y, and θ can be used as the final position information and angle value of each target object.

[0157] Based on the above embodiments, the embodiments of the present application comprehensively illustrate the following feasible implementation manners for example. For reference, as Figure 2 shown, Figure 2 Figure 2 is an exemplary template image I in the object detection method based on template matching of the present application tpl , and the size of this template image is W tpl = 322, H tpl = 255. Figure 3 Figure 3 is an exemplary mask image in the object detection method based on template matching of the present application. The size of this mask image is the same as that of the template image, and all pixel values of the mask image are 1.

[0158] First, the corresponding initial pyramid layer number can be obtained according to the size of the template image. For example, in this embodiment, the sizes of each layer in the template image pyramid corresponding to the initial pyramid layer number are W1 = 322, H1 = 255, W2 = 161, H2 = 127, W3 = 80, H3 = 63, W4 = 40, H4 = 31, W5 = 20, H4 = 15, W6 = 10, H6 = 7 respectively. Among them, the size threshold of the present application can be set to 12, and the size of the top - layer template image W6 < 12 and H6 < 12. Therefore, the initial pyramid layer number is determined to be 6.

[0159] Secondly, the initial pyramid layer number can be adjusted according to the number of valid pixels in the mask image. In this embodiment, when the initial pyramid layer number is 6, the number of valid pixels cnt notzero = W6 * H6 = 70 of the top - layer mask image. If the pixel number threshold is 64, cnt notzero ≥ 64. Therefore, there is no need to adjust the initial pyramid layer number.

[0160] Then, the pyramid layer number can be adjusted according to the target information entropy corresponding to the top - layer template image. For reference, as Figure 4 shown, Figure 4 Figure 4 is an exemplary top - layer template image in the object detection method based on template matching of the present application, and its one - dimensional gray entropy is 5.58. Then, the gradient magnitude and gradient direction of the top - layer template image are calculated using the sobel operator. For reference, as Figure 5 and Figure 6 shown, Figure 5 Figure 5 is an exemplary top - layer template gradient magnitude map in the object detection method based on template matching of the present application, Figure 6This is an exemplary top - layer template gradient direction map in the object detection method based on template matching of this application. Similarly, calculate its one - dimensional entropy to be 5.81 and 5.84 respectively, and finally calculate the sum to obtain the object information entropy H = 17.23. If the preset information entropy threshold is 5 and the object information entropy H is greater than 5, this indicates that there is still sufficient information in the top - layer template image of the pyramid. Therefore, it is not necessary to adjust the number of pyramid layers. Finally, the initial number of pyramid layers can be determined as the target number of pyramid layers pyr = 6. According to the target number of pyramid layers, a template image pyramid and a mask image pyramid are established respectively. An example can be referred to as Figure 7 as shown Figure 7 This is an example scenario diagram for constructing an image pyramid in the object detection method based on template matching of this application.

[0161] Furthermore, rotate the top - layer template image within the set angle range and calculate relevant information. In this embodiment, the set template angle range can be [0, 360]. According to the template size, the top - layer rotation step size is calculated to be 11 degrees, which is equivalent to rotating the top - layer template image and the top - layer mask image every 11 degrees within the angle range [0, 360], and calculating the total number of valid pixels, the average gray value of the rotated template image, and the cumulative value of the squared gray values of the rotated template image in each rotated template image. Thus, the template training is completed. Among them, partial rotation results of the top - layer template image and the top - layer mask image can be referred to as Figure 8 as shown Figure 8 This is an example scenario diagram for image rotation processing in the object detection method based on template matching of this application.

[0162] During the image matching process, an example can be referred to as Figure 9 as shown Figure 9 This is an exemplary object image in the object detection method based on template matching of this application. The object detection method of this application can then slide the template image on this object image for matching to determine the target object in the object image. In the matching stage, first establish an object image pyramid, and its number of pyramid layers is the same as that of the template image pyramid. An example can be referred to as Figure 10 as shown Figure 10 This is an exemplary object image pyramid in the object detection method based on template matching of this application.

[0163] The matching starts from the top layer of the pyramid. Calculate the normalized correlation measure between each rotated template image in the top layer and the top - layer object image, and obtain the maximum value of the normalized correlation measure at the same coordinates and its corresponding angle. The calculation results can be referred to as Figure 11 and Figure 12 as shown Figure 11 The result diagram of the top - layer normalized cross - correlation measure in the object detection method based on template matching of this application, Figure 12It is the angle graph corresponding to the maximum value of the top-level normalized cross-correlation metric in the object detection method based on template matching of the present application. The area enclosed by the dashed box is the true position of the object to be matched. It can be seen from this that in this embodiment, the normalized correlation metric score (object similarity score) of the object to be matched at the top level is 0.8188, the corresponding matching angle is 11 degrees, and the coordinates are (3, 5). In this way, one of the objects with the maximum normalized correlation metric score is cand 6_0 =(3, 5, 11, 0.8188).

[0164] Then, starting from the 5th layer to the 1st layer in the image pyramid, the relevant information of the object is updated layer by layer. Exemplarily, reference can be made to Figure 13 as shown Figure 13 It is a schematic diagram of pyramid iterative matching in the object detection method based on template matching of the present application. For the object with the maximum normalized correlation metric score in this embodiment, the relevant information at each layer of the pyramid is as follows:

[0165] Layer 5 cand 5_0 =(5, 9, 16.3565, 0.9142)

[0166] Layer 4 cand 4_0 =(9, 18, 16.3565, 0.8715)

[0167] Layer 3 cand 2_0 =(18, 36, 15.0174, 0.9224)

[0168] Layer 2 cand 2_0 =(34, 70, 15.0174, 0.9849)

[0169] Layer 1 cand 1_0 =(67, 139, 15.0174, 0.9923)

[0170] Finally, high-precision module calculation is performed, using both the angle and position information to participate in fitting, and improving the accuracy of both the matching position and the matching angle at the same time. The calculation result of the high-precision module in this embodiment is cand0 = (67.049, 139.349, 15.0008, 0.9923). The matching result can be referred to Figure 14 as shown Figure 14 It is a schematic diagram of the matching result in the object detection method based on template matching of the present application. The matching object can be converted into the centroid form to represent the relevant information of the object, which is (254.549, 303.349, 15.0008, 0.9923). Thus, the entire object matching process is completed.

[0171] It should be further noted that the execution entity of the object detection method based on template matching can be an object detection device based on template matching. For example, the object detection method based on template matching can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the object detection method based on template matching can be implemented by a processor calling computer-readable instructions stored in a memory.

[0172] Figure 15 is a block diagram of an object detection device based on template matching shown in an exemplary embodiment of the present application. As Figure 15 shown, the exemplary object detection device 300 based on template matching includes: an acquisition module 310, an adjustment module 320, a pyramid construction module 330, and a matching module 340. Specifically:

[0173] The acquisition module 310 is configured to acquire an initial pyramid level corresponding to a template image.

[0174] The adjustment module 320 is configured to adjust the initial pyramid level according to the mask information to obtain a target pyramid level in response to the mask information of the mask image corresponding to the template image not satisfying a preset condition.

[0175] The pyramid construction module 330 is configured to construct a template image pyramid of the template image according to the target pyramid level, and construct a target image pyramid of the target image to be matched according to the target pyramid level.

[0176] The matching module 340 is configured to match the top template image in the template image pyramid with the top target image in the target image pyramid to obtain a target object in the top target image.

[0177] In the exemplary template matching-based target detection device, the initial pyramid level corresponding to the template image is obtained; the mask information of the mask image corresponding to the template image is compared with a preset condition, and when the mask information does not meet the preset condition, the initial pyramid level can be adjusted according to the mask information to obtain an adaptive target pyramid level; a template image pyramid of the template image is constructed according to the target pyramid level, and a target image pyramid of the target image to be matched is constructed according to the target pyramid level; the top template image in the template image pyramid is matched with the top target image in the target image pyramid to obtain the target object in the top target image, thereby reducing the computational amount in the image matching process, improving the efficiency of image matching, and the pyramid level based on adaptive adjustment can also ensure the target detection accuracy.

[0178] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept. The specific ways in which each module and unit perform operations have been described in detail in the method embodiment, and will not be repeated here. In practical applications, the device provided in the above embodiment can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here.

[0179] Among them, the functions of each module can be referred to in the embodiment of the template matching-based target detection method, and will not be repeated here.

[0180] Please refer to Figure 16 , Figure 16 which is a schematic structural diagram of an embodiment of an electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is used to execute the program instructions stored in the memory 101 to implement the steps in any of the above embodiments of the template matching-based target detection method. In a specific implementation scenario, the electronic device 100 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 100 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited here.

[0181] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above embodiments of the object detection method based on template matching. The processor 102 can also be referred to as a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 can also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Additionally, the processor 102 can be implemented jointly by integrated circuit chips.

[0182] In this exemplary electronic device, by obtaining the initial pyramid level corresponding to the template image; comparing the mask information of the mask image corresponding to the template image with a preset condition, when the mask information does not meet the preset condition, the initial pyramid level can be adjusted according to the mask information to obtain an adaptive target pyramid level; constructing a template image pyramid of the template image according to the target pyramid level, and constructing a target image pyramid of the target image to be matched according to the target pyramid level; matching the top template image in the template image pyramid with the top target image in the target image pyramid to obtain the target object in the top target image, thereby reducing the computational amount in the image matching process, improving the efficiency of image matching, and also ensuring the object detection accuracy based on the adaptively adjusted pyramid level.

[0183] Please refer to Figure 17 , Figure 17 is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be run by a processor, and the program instructions 111 are used to implement the steps in any of the above embodiments of the object detection method based on template matching.

[0184] In this exemplary storage medium, by running the program instructions in the storage medium, the initial number of pyramid levels corresponding to the template image is obtained; the mask information of the mask image corresponding to the template image is compared with a preset condition, and when the mask information does not meet the preset condition, the initial number of pyramid levels can be adjusted according to the mask information to obtain an adaptive target number of pyramid levels; a template image pyramid of the template image is constructed according to the target number of pyramid levels, and a target image pyramid of the target image to be matched is constructed according to the target number of pyramid levels; the top template image in the template image pyramid is matched with the top target image in the target image pyramid to obtain the target object in the top target image, thereby reducing the computational complexity in the image matching process, improving the efficiency of image matching, and also ensuring the target detection accuracy based on the adaptively adjusted number of pyramid levels.

[0185] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0186] The descriptions of the above embodiments tend to emphasize the differences between the embodiments. The similarities or similarities between them can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0187] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0188] In addition, each functional unit in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. If the integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

Claims

1. A target detection method based on template matching, characterized in that, The method includes: Obtaining an initial pyramid level corresponding to a template image; In response to the mask information of the mask image corresponding to the template image not satisfying a preset condition, adjusting the initial pyramid level according to the mask information to obtain a target pyramid level; Constructing a template image pyramid of the template image according to the target pyramid level, and constructing a target image pyramid of a target image to be matched according to the target pyramid level; Matching a top-level template image in the template image pyramid with a top-level target image in the target image pyramid to obtain a target object in the top-level target image.

2. The method according to claim 1, wherein The obtaining an initial pyramid level corresponding to a template image includes: Obtaining a template size of the template image; Performing successive downsampling processing on the template image according to the template size to obtain a downsampled template image; In response to the current size of the downsampled template image being smaller than a size threshold, determining the initial pyramid level according to the number of sampling times.

3. The method according to claim 1, characterized in that, The adjusting the initial pyramid level according to the mask information to obtain a target pyramid level includes: Obtaining the number of valid pixels in the mask information; In response to the number of valid pixels being smaller than a pixel number threshold, performing a reduction process on the initial pyramid level to obtain the target pyramid level.

4. The method according to claim 1, wherein The adjusting the initial pyramid level according to the mask information to obtain a target pyramid level includes: Adjusting the initial pyramid level until the mask information satisfies the preset condition to obtain a pending pyramid level; Determining a target information entropy corresponding to the template image and the mask image according to the pending pyramid level; In response to the target information entropy being smaller than an information entropy threshold, performing a reduction process on the pending pyramid level to obtain the target pyramid level.

5. The method according to claim 4, wherein The determining a target information entropy corresponding to the template image and the mask image according to the pending pyramid level includes: Determining a pending top-level template image corresponding to the template image and a pending top-level mask image corresponding to the mask image according to the pending pyramid level; Determining a gray entropy of the pending top-level template image according to the valid pixels represented by the pending top-level mask image and the gray information of the pending top-level template image; Performing gradient calculation on the pending top-level template image to obtain a gradient amplitude entropy and a gradient direction entropy of the pending top-level template image; Determining the target information entropy according to the gray entropy, the gradient amplitude entropy, and the gradient direction entropy.

6. The method according to claim 1, characterized in that, The matching a top-level template image in the template image pyramid with a top-level target image in the target image pyramid to obtain a target object in the top-level target image includes: Performing sliding matching of the top-level template image in the top-level target image according to a preset sliding step length, and respectively calculating a target similarity score between the top-level target image and the top-level template image at each sliding position corresponding to the preset sliding step length; Determining a target object from the top-level target image according to the target similarity score.

7. The method according to claim 6, wherein The top - layer template image includes multiple rotated template images, which are obtained through image rotation processing between different rotated template images. Sliding - matching the top - layer template image in the top - layer target image according to a preset sliding step length, and calculating the target similarity score between the top - layer target image and the top - layer template image at each sliding position corresponding to the preset sliding step length, includes: Sliding - matching the top - layer target image with each rotated template image respectively to obtain a corresponding plurality of similarity score maps; Performing a merging process based on the maximum similarity scores in each similarity score map to obtain a target similarity score map, and the target similarity score map includes the target similarity score.

8. The method according to claim 6, wherein Determining the target object from the top - layer target image according to the target similarity score includes: Determining the pixel points with the target similarity score greater than the similarity threshold as the relevant coordinate points of the target object; Determining the position information of the target object according to the position information of the relevant coordinate points.

9. An electronic device, characterized in that, It includes a memory and a processor, and the processor is used to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images