Target positioning method, apparatus, device, and storage medium

Abstract

This disclosure provides a target localization method, apparatus, device, and storage medium. The method includes: acquiring multiple sub-images of an image to be processed and the gradient features of each sub-image; for each sub-image, determining a sub-template image with the same size and resolution to obtain multiple image pairs; determining the similarity between the sub-template image and the sub-image to be processed in the image pair based on the gradient features of the sub-template image and the sub-image to be processed; selecting candidate image pairs whose similarity meets a threshold, and determining the location information of the target object in the image to be processed based on the location information of the standard object in the sub-template image of all candidate image pairs. Using gradient features as the basis for comparing the similarity between the sub-image and the sub-template image is more robust than grayscale information, resulting in a more accurate target object location.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a target positioning method, apparatus, device, and storage medium. Background Technology

[0002] In industrial manufacturing and intelligent inspection, accurate target localization is a core component for automating processes and improving production efficiency and product quality. The key requirement is to quickly and accurately identify the location of target objects from acquired image data. Related target localization technologies typically employ grayscale-based modeling and matching methods, which locate the target image by comparing its grayscale values ​​with those of a target template image. However, grayscale information is easily affected by lighting conditions, shooting angle, and other shooting conditions. For the same target, the grayscale values ​​of the obtained test images differ under different shooting conditions. Therefore, relying solely on grayscale information for template matching suffers from severely insufficient matching accuracy. Summary of the Invention

[0003] This disclosure provides a target positioning method, apparatus, device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] A first aspect of this disclosure provides a target localization method, the method comprising: The process involves acquiring multiple sub-images of the image to be processed and the gradient features of each sub-image. The multiple sub-images to be processed include N groups of sub-images to be processed, and each group of sub-images to be processed includes M sub-images of the same size but different resolutions. For each sub-image to be processed, determine a sub-template image with the same size and resolution to obtain multiple image pairs; The similarity between the neutron template image and the sub-image to be processed is determined based on the gradient features of the neutron template image and the gradient features of the sub-image to be processed. Candidate image pairs whose similarity meets the threshold are selected, and the position information of the target object in the image to be processed is determined based on the position information of the standard object in the sub-template image of all candidate image pairs.

[0005] In one possible implementation, obtaining the gradient features of the sub-template image or the gradient features of the sub-image to be processed includes: Determine the gradient magnitude and gradient direction of multiple image channels for each pixel in the image, and use the median gradient magnitude as the gradient magnitude of that pixel; The gradient direction corresponding to the median gradient magnitude is taken as the gradient direction of the pixel. A target region centered on the pixel is determined, and the gradient direction of the pixel is updated using the gradient directions of other pixels within the target region. After quantization, the updated gradient direction is combined with the gradient magnitude of the pixel to form the gradient feature of the pixel, and the gradient features of all pixels constitute the gradient feature of the image.

[0006] In one possible implementation, determining the similarity between the neutron template image and the sub-image to be processed based on the gradient features of the neutron template image and the gradient features of the sub-image to be processed includes: For each pixel in the sub-template image, determine the matching region corresponding to that pixel in the sub-image to be processed. The matching region is the region centered on the corresponding pixel in the sub-image to be processed. The similarity between the gradient feature of the pixel and the gradient features of all pixels in the matching region is compared to obtain the similarity between the pixel and the matching region. The similarity between all pixels and the matching region is summed to obtain the similarity between the sub-template image and the sub-image to be processed in the image pair.

[0007] In one possible implementation, determining the position information of the target object in the image to be processed based on the position information of the standard objects in the neutron template image of all candidate image pairs includes: Determine the center point of the standard object in the neutron template image of all candidate image pairs; Based on the coordinates of all center points, the coordinates of the center point of the target object in the image to be processed are determined to characterize the position information of the target object.

[0008] In one possible implementation, determining the coordinates of the target object's center point in the image to be processed based on the coordinates of all center points includes: Assign corresponding weights to the coordinates of all center points; The coordinates of all center points are added together with the products of their corresponding weights to obtain a weighted fusion result. The ratio of the weighted fusion result to the sum of all weights is then determined to obtain the coordinates of the center point of the target object in the image to be processed.

[0009] In one possible implementation, the quantization of the updated gradient direction includes: For each pixel, the direction interval to which the updated gradient direction of the pixel belongs is determined according to the quantization rule. The quantization rule is used to divide the continuous circumferential direction represented by the gradient direction into multiple direction intervals at equal intervals, and assign an interval identifier code to each direction interval. By combining the interval identifier codes corresponding to each directional interval of the pixel, an interval identifier code sequence is obtained.

[0010] A second aspect of this disclosure provides a target positioning device, the device comprising: The image processing module is used to acquire multiple sub-images of the image to be processed and the gradient features of each sub-image. The multiple sub-images to be processed include N sub-image groups, and each sub-image group includes M sub-images of the same size but different resolutions. The similarity comparison module is used to identify sub-template images with the same size and resolution for each sub-template image, thus obtaining multiple image pairs. The similarity comparison module is further configured to determine the similarity between the image pair neutron template image and the image to be processed based on the gradient features of the image pair neutron template image and the gradient features of the sub-image to be processed. The target recognition module is used to select candidate image pairs whose similarity meets the threshold, and determine the position information of the target object in the image to be processed based on the position information of the standard object in the sub-template image of all candidate image pairs.

[0011] In one embodiment, the similarity comparison module is further configured to determine, for each pixel of the sub-template image, the matching region corresponding to that pixel in the sub-image to be processed, wherein the matching region is a region centered on the corresponding pixel in the sub-image to be processed; The similarity between the gradient feature of the pixel and the gradient features of all pixels in the target region is compared to obtain the similarity between the pixel and the matching region. The similarity between all pixels and the matching region is summed to obtain the similarity between the sub-template image and the sub-image to be processed in the image pair.

[0012] A third aspect of this disclosure provides an electronic device comprising: At least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the target localization method described in any of the preceding claims.

[0013] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program for performing the target localization method described in any one of the preceding claims.

[0014] This disclosure discloses a target localization method. When locating a target in an image to be processed, the method involves adjusting the size of the image to obtain multiple sets of sub-images to be processed. Each sub-image in the set has the same size but different resolution, and the gradient features of each sub-image are determined. For each sub-image, a matching sub-template image is determined, i.e., a sub-template image with the same size and resolution as the original image, resulting in multiple image pairs. For each image pair, the gradient feature similarity between the sub-template image and the sub-image to be processed is compared to select multiple target sub-template images. The target object location information in the image to be processed is calculated using the standard object location information of the multiple target sub-template images. Compared to using grayscale information of images for similarity comparison, gradient features are more robust and less affected by external factors such as shooting time and lighting conditions. Using them as a benchmark for comparing the sub-image to be processed with the sub-template image can effectively determine the accurate sub-template image, thereby ensuring that the target object position information determined based on the standard object position information in the sub-template image is more accurate.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0016] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0017] Figure 1 A schematic diagram illustrating the implementation flow of a target localization method according to an embodiment of this disclosure is shown; Figure 2 A schematic diagram of an image to be processed according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of a design document according to an embodiment of this disclosure is shown; Figure 4 A schematic diagram of a sub-template image according to an embodiment of the present disclosure is shown; Figure 5 Another sub-template image schematic diagram of an embodiment of this disclosure is shown; Figure 6 This diagram illustrates a visualization result of the spatial distribution of gradient magnitude according to an embodiment of the present disclosure. Figure 7 This diagram illustrates a visualization result of a discrete gradient direction distribution according to an embodiment of the present disclosure. Figure 8 A schematic diagram of a target positioning device according to an embodiment of the present disclosure is shown; Figure 9 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0018] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0019] This disclosure provides a target localization method, such as... Figure 1 As shown, the method includes: S101. Obtain multiple sub-images to be processed of the image to be processed and the gradient features of each sub-image to be processed. The multiple sub-images to be processed include N groups of sub-images to be processed, and each group of sub-images to be processed includes M sub-images to be processed with the same size but different resolutions. In this step, such as Figure 2 As shown, the image to be processed is the target object image to be located, specifically the screw hole image 201 whose center point location information needs to be determined. The sub-images to be processed are images obtained by sequentially adjusting the size and resolution of the original image. It should be noted that multiple sub-images to be processed are divided into N groups based on size, with each group containing sub-images of different resolutions. The nearest neighbor difference method is used to adjust the size of the images to be processed to between 0.7 and 1.3 times, resulting in a total of 60 groups of sub-images to be processed.

[0020] To further improve the accuracy of subsequent target localization, this step also adjusts the resolution of each group of sub-images to be processed. Each image after size adjustment is downsampled, with its resolution adjusted to 1 / 4 and 1 / 8 of the original resolution, respectively. This results in 60 groups of sub-images to be processed, each group containing sub-images of the same size but different resolutions (original resolution, 1 / 4 of the original resolution, and 1 / 8 of the original resolution), totaling 180 sub-images. Gradient features are then extracted from these 180 sub-images, yielding 180 gradient features.

[0021] S102. For each sub-image to be processed, determine a sub-template image with the same size and resolution to obtain multiple image pairs; In this step, the sub-template image is the image obtained after adjusting the size and resolution of the template image. The template image is an image of a standard object recorded in the design document, specifically the screw holes on the laptop's C-shell, i.e., the screw holes on the panel shell containing the keyboard and touchpad. For example... Figure 3 As shown, the corresponding design document contains images of the standard screw holes on the laptop's C-shell. In addition, the design document also records information such as the location and size of the standard screw holes.

[0022] It should be noted that there are multiple template images, and these multiple sub-template images are divided into N groups based on size. The sub-template images in each group have different resolutions. That is, the sub-template images in the i-th sub-template image group have the same size as the sub-images to be processed in the i-th sub-image group to be processed, and their resolutions correspond one-to-one.

[0023] For example, the template image can be scaled from its original size to 0.7 times using the adjacent difference method to obtain an image of the corresponding size, such as... Figure 4 As shown, let's denote the first sub-template image 401. We perform downsampling on the first sub-template image 401, adjusting its resolution to 1 / 4 and 1 / 8 of the original, resulting in the first group of sub-template images. This first group includes three sub-template images of the same size but different resolutions. Based on this, other groups of sub-template images are generated with a step size of 0.01. Specifically, the template images are scaled from their original size to 0.71 times using the adjacent difference method, resulting in images of the corresponding size, denoted as the second sub-template images. The second sub-template images are then adjusted to the same resolution as the first sub-template images, resulting in the second group of sub-template images. For example, as... Figure 5 As shown, the template image is adjusted from its original size to 1.3 times using the adjacent difference method, resulting in an image of the corresponding size, denoted as the sixtieth sub-template image 501. The sixtieth sub-template image 501 is then adjusted to the same resolution as the first sub-template image, resulting in the sixtieth group of sub-template images. The processing method for other groups of sub-template images is the same as that for the first, second, and sixtieth groups, and so on, until 60 groups of sub-template images (a total of 180 sub-template images) are obtained, covering different sizes and resolutions. The adjacent difference method ensures that the corresponding standard object in the resized image, i.e., the circular screw hole, remains unchanged. Using this as a template ensures the accuracy of the subsequent matching results with the screw holes in the image to be processed. The corresponding adjacent difference formula is as follows:

[0024] In the formula, The target pixel location (coordinates in the resized image). For the target pixel value, , , and These are the coordinates of the top-left, top-right, bottom-left, and bottom-right reference points of the pixel corresponding to the target pixel in the original image. = , = ,and , It varies within the range of (0, 1).

[0025] It's also worth noting that the reason for setting 180 sub-template images in this step is that screw holes have tolerances in actual manufacturing. Even if manufactured strictly according to the standard information recorded in the design document, it is difficult to guarantee that the size of each screw hole is the same. Therefore, this step sets multiple sub-template images, covering different sizes, to provide corresponding matching templates for all screw holes with reasonable errors. When locating screw holes later, the corresponding sub-template image can be found. At the same time, the image resolution within each group is different, simulating the appearance of the target object at different levels of clarity. Even if the image to be processed is relatively blurry, the corresponding sub-template image can be found among the multiple sub-template images, serving as the basis for determining the position information of the target object to be located, ensuring that the target object's position information can be accurately determined later. In addition, in this step, for each sub-template image, the corresponding sub-image to be processed can be determined based on the size parameters and resolution parameters, thus obtaining multiple image pairs. For example, for a set of sub-template images whose size parameter is adjusted by a factor of 1.2, one sub-template image in this set with a resolution of 1 / 8 of the original resolution, and another sub-image in a set of images to be processed with the same size parameter adjusted by a factor of 1.2 and a resolution of 1 / 8 of the original resolution, constitute an image pair. The gradient feature similarity of this image pair is calculated to determine whether to use the sub-template image in this pair as the sub-template image for subsequently determining the location information of the target object in the image to be processed. That is, this step performs image matching in a one-to-one correspondence based on size and resolution as indices. The matching processes of different groups are independent of each other and can be processed in parallel, effectively improving computational efficiency.

[0026] S103. Determine the similarity between the neutron template image and the sub-image to be processed based on the gradient features of the neutron template image and the gradient features of the sub-image to be processed. In this step, the similarity between image pairs is calculated based on the gradient features of the sub-template image and the sub-image to be processed. Compared to calculating similarity using the grayscale information of the images, this can effectively improve the positioning accuracy. This is because the grayscale values ​​of an image are easily affected by lighting, shooting angle, etc. The same screw hole may have different grayscale values ​​in the images to be processed under different shooting conditions. Directly comparing the grayscale values ​​of the images to be processed with those of the template image results in inaccurate matching. Gradient features (gradient magnitude and gradient direction) are more robust and reliable, such as... Figure 6 and Figure 7 As shown, it represents the contrast between light and dark at the edge of the screw hole. Even when the screw hole images are taken under different lighting conditions, the transition characteristics from light to dark at the screw hole edge are stable. Using this as a benchmark for comparing the image to be processed with the template image can effectively determine the accurate template image, ensuring that the target object position information determined based on the standard object position information in the template image is more accurate.

[0027] S104. Select candidate image pairs whose similarity meets the threshold, and determine the position information of the target object in the image to be processed based on the position information of the standard object in the sub-template image of all candidate image pairs.

[0028] In this step, the similarity of each image pair calculated in step S103 is sorted from high to low, and corresponding candidate image pairs are selected based on a similarity threshold. For example, based on the similarity threshold, 30 sub-template images that meet the threshold criteria are selected from 180 sub-template images. The location information of the standard object in these 30 sub-template images can be combined to obtain the location information of the target object in the image to be processed. By comprehensively determining the location information of the target object through multiple sub-template images, the possibility of incorrect positioning due to accidental factors when using a single template image as a positioning reference is reduced, resulting in more accurate positioning results.

[0029] This disclosure provides a target localization method. When locating a target in an image to be processed, the method involves adjusting the size of the image to obtain multiple groups of sub-images to be processed. Each sub-image group contains sub-images of the same size but different resolutions, and the gradient features of each sub-image are determined. For each sub-image, a matching sub-template image is determined, i.e., a sub-template image with the same size and resolution as the original image, resulting in multiple image pairs. For each image pair, the gradient feature similarity between the sub-template image and the sub-image to be processed is compared to select multiple target sub-template images. The target object location information in the image to be processed is calculated using the standard object location information of the multiple target sub-template images. Compared to using grayscale information of images for similarity comparison, gradient features are more robust and less affected by external factors such as shooting time and lighting conditions. Using them as a benchmark for comparing the sub-image to be processed with the sub-template image can effectively determine the accurate template image, thereby ensuring that the target object position information determined based on the standard object position information in the template image is more accurate.

[0030] In one possible implementation, obtaining the gradient features of the sub-template image or the gradient features of the sub-image to be processed includes: Determine the gradient magnitude and gradient direction of multiple image channels for each pixel in the image, and use the median gradient magnitude as the gradient magnitude of that pixel; The gradient direction corresponding to the median gradient magnitude is taken as the gradient direction of that pixel. The target region centered on the pixel is determined, and the gradient direction of the pixel is updated using the gradient directions of other pixels in the target region. After quantization, the updated gradient direction and the gradient magnitude of the pixel are combined to form the gradient feature of the pixel. The gradient features of all pixels constitute the gradient feature of the image.

[0031] In this embodiment, the gradient features of both the sub-template image and the sub-image to be processed are extracted using the following method. The gradient features include gradient magnitude and gradient direction; the gradient magnitude and gradient direction of each pixel in the image are calculated based on the horizontal and vertical gradient values ​​of that pixel.

[0032] The formula for calculating the horizontal gradient value is:

[0033] In the formula The coordinates on the image are The grayscale value of the pixel; The formula for calculating the gradient value in the vertical direction is:

[0034] The calculation formula for the gradient magnitude is as follows:

[0035] The calculation formula for the gradient direction is as follows:

[0036] It should be noted that each pixel point in the image corresponds to three channels: red (R), green (G), and blue (B). In this embodiment, the corresponding gradient magnitudes and gradient directions are calculated for the three channels of the pixel point according to the above gradient magnitude formula and gradient direction formula respectively.

[0037] For example, for pixel point P, the gradient magnitude corresponding to its R channel is calculated based on the gray values of the R channels of the surrounding pixel points. That is, for point P(x, y), the horizontal gradient magnitude of its R channel is calculated using the gray values of the R channels of the points (x + 1, y + 1), (x + 1, y), (x + 1, y - 1), (x - 1, y + 1), (x - 1, y), and (x - 1, y - 1). Correspondingly, the vertical gradient magnitude of its R channel is calculated based on the gray values of the R channels of the points (x - 1, y + 1), (x, y + 1), (x + 1, y + 1), (x, y - 1), (x + 1, y - 1), and (x - 1, y - 1). Substitute the horizontal gradient magnitude and the vertical gradient magnitude into the formula and the formula to calculate the gradient magnitude GR and gradient direction ori_R of the R channel of point P. Similarly, the gradient magnitude GG and gradient direction ori_G of the G channel of point P, and the gradient magnitude GB and gradient direction ori_B of the B channel are determined in the same way as the R channel, and the repetitive parts will not be elaborated.

[0038] Furthermore, sort the gradient magnitudes of the three channels calculated above to determine the median gradient magnitude. For example, if GR < GB < GG, then for pixel point P, the gradient magnitude GB of its B channel is the median gradient magnitude. Selecting the median gradient magnitude as the final gradient magnitude can effectively filter Gaussian noise. Correspondingly, the gradient direction ori_B of the B channel is the gradient direction determined for pixel point P. The specific calculation formula is as follows: And so on, the corresponding gradient magnitude and gradient direction of each pixel point on the image are determined in the above manner, and the repetitive parts will not be elaborated.

[0039] It should be noted that when determining the gradient direction of each pixel in the image, it is also necessary to determine whether the median gradient magnitude of that pixel exceeds a set magnitude threshold. In this embodiment, the magnitude threshold is preferably 35. If the median gradient magnitude is less than or equal to 35, it may be noise, and the corresponding gradient direction should be discarded. If the median gradient magnitude is greater than 35, the corresponding gradient direction is retained for the next step of processing. That is, after determining the gradient direction of each pixel in the image, the gradient direction of that pixel is updated based on the gradient directions of other pixels in the target neighborhood of that pixel.

[0040] It should also be noted that the target neighborhood of a pixel is a target region on the image centered on that pixel. The gradient direction of that pixel is updated using the gradient directions of other pixels within the target region. In this embodiment, the gradient direction is based on pixel 5. The gradient direction is updated for pixels within a 5-neighborhood. For example, for point P, its 5-neighborhood gradient direction is updated. A 5-neighborhood refers to a target region where point P(x, y) is the center point, and the corresponding pixel coordinates are: top-left (x-2, y+2), bottom-left (x-2, y-2), top-right (x+2, y+2), and bottom-right (x+2, y-2). The gradient direction of the center pixel is updated based on the gradient directions of other pixels within the neighborhood (excluding the center point). For example, if the gradient direction of the center point is horizontal to the left (0°), and other pixels in its neighborhood also have vertical directions upward (90°) and horizontal directions to the right (180°), then the updated gradient direction of the center point includes three directions: horizontal to the left (0°), vertical directions upward (90°), and horizontal directions to the right (180°).

[0041] In addition, this embodiment also performs quantization processing on the updated gradient direction. Accordingly, the gradient direction is converted into a corresponding encoding sequence according to a preset quantization rule, which serves as the final gradient feature of the pixel. The final gradient features of all pixels in the image constitute the final gradient feature of the image. The gradient feature is used as the basis for subsequent similarity matching. Compared with using absolute gray values ​​for image similarity comparison, the gradient feature is less affected by external factors (lighting). The brightness of the light has little impact on the actual screw hole detection accuracy, thereby ensuring the accuracy of screw hole positioning.

[0042] In one possible implementation, the updated gradient direction is quantized, including: For each pixel, the direction interval to which the updated gradient direction of the pixel belongs is determined according to the quantization rule. The quantization rule is used to divide the continuous circumferential direction represented by the gradient direction into multiple direction intervals at equal intervals, and assign an interval identifier code to each direction interval. By combining the interval identifier codes corresponding to each directional interval of the pixel, an interval identifier code sequence is obtained.

[0043] In this embodiment, the quantization rule is to divide the gradient direction from 0 to 360° into 10 equal directional intervals, each interval being 36° in size, and assigning a corresponding interval identifier code. Preferably, this embodiment uses 10-bit binary codes to quantize the corresponding directional intervals. For example, [0°, 36°) corresponds to directional interval 0. If the updated gradient direction of a pixel belongs to this interval, the corresponding interval identifier code is 1; otherwise, it is 0. Similarly, [36°, 72°) corresponds to directional interval 1. If the updated gradient direction of a pixel belongs to this interval, the corresponding interval identifier code is 1; otherwise, it is 0, and so on, to obtain the interval identifier codes corresponding to the 10 directional intervals.

[0044] For example, for pixel M, its initial gradient direction, determined based on the median gradient magnitude, is 20°, belonging to direction interval 0. Therefore, the pixel's direction interval 0 is recorded as 1. Simultaneously, this pixel is based on its 5... After updating other pixels in the 5-neighborhood, the gradient direction is augmented with 40° and 90° gradient directions corresponding to other pixels in the neighborhood, belonging to direction interval 1 and direction interval 2 respectively. Therefore, both direction interval 1 and direction interval 2 bits for this pixel are recorded as 1, resulting in a final interval identifier encoding sequence of 00 0000 0111 for pixel M. Similarly, each pixel in the image can obtain a corresponding interval identifier encoding sequence. This embodiment converts the gradient direction from a continuous angle to a discrete direction. Compared to a continuous angle, the quantized gradient direction is less sensitive to interference factors such as noise and changes in illumination. For example, different illumination conditions can cause slight changes in the continuous direction, such as changing from 44.9° to 45.1°, which are two different values ​​in the continuous direction. However, after quantization into the corresponding direction interval, all values ​​belong to direction interval 2, resulting in stronger robustness. Subsequent matching results based on this are more accurate and less affected by interference such as changes in illumination conditions.

[0045] In one possible implementation, determining the similarity between the neutron template image and the sub-image to be processed based on the gradient features of the neutron template image and the gradient features of the sub-image to be processed includes: For each pixel in the sub-template image, determine the matching region corresponding to that pixel in the sub-image to be processed. The matching region is the region centered on the corresponding pixel in the sub-image to be processed. The similarity between the gradient feature of the pixel and the gradient features of all pixels in the matching region is obtained by comparing the gradient features of the pixel with those of the matching region. The similarity between all pixels and the matching region is summed to obtain the similarity between the sub-template image and the sub-image to be processed in the image pair.

[0046] In this embodiment, sub-template images that can be used to locate screw holes in the image to be processed are selected by comparing the feature similarity between the sub-template image and the sub-image to be processed in the image pair. That is, the feature similarity comparison is performed between sub-template images and sub-images to be processed of the same size and resolution.

[0047] The similarity calculation formula is as follows:

[0048] In the formula, Let I represent the similarity between the gradient features of the sub-image to be processed and the gradient features of the sub-template image, and T represent the sub-template image. , For the image by position Centered on and of size The neighborhood, Indicates 3 3-neighborhood Let r be the gradient direction at position r in the sub-template image. Let t be the gradient direction at position t in the sub-image to be processed. , To select the pixel most similar to the sub-template image within the neighborhood of the sub-image to be processed, the pixel with the highest similarity is selected. This represents the sum of the maximum similarity across all feature locations of the template, yielding the overall similarity score.

[0049] For each pixel in the sub-template image, calculate its gradient feature and the 3D coordinates centered at the corresponding pixel in the sub-image to be processed. The similarity between the sub-template image and the sub-image to be processed is obtained by combining the gradient feature similarity of all pixels in the neighborhood and the similarity results of all pixels in the sub-template image. In this embodiment, the gradient features of each sub-template image are pre-calculated and stored based on the template image in the design document. During the online matching stage, only the quantized gradient features of the sub-image to be processed need to be calculated, resulting in a relatively faster matching speed. Furthermore, the similarity comparison is performed between image pairs of the same size and resolution, allowing multiple image pairs to be processed in parallel, resulting in a relatively fast overall processing speed. In addition, related image detection methods, such as corner detection, although fast, can only detect corner positions. However, the circular screw hole edge corner features detected in this embodiment are few, making it difficult to accurately locate the circular screw hole using corner detection. This embodiment performs similarity matching based on the quantized gradient direction, which is adapted to the circular screw hole. The gradient direction is naturally perpendicular to the circular edge, making it more stable than features such as corner points. The sub-template image matched in this way is more accurate, and the center coordinates of the screw hole to be tested calculated based on the sub-template image are more precise.

[0050] In one possible implementation, the location information of the target object in the image to be processed is determined based on the location information of the standard objects in the neutron template image of all candidate image pairs, including: Determine the center point of the standard object in the neutron template image of all candidate image pairs; Based on the coordinates of all center points, determine the coordinates of the center point of the target object in the image to be processed, so as to characterize the position information of the target object.

[0051] In this embodiment, the similarity between the sub-template images and the sub-image to be processed in the image pair is sorted from high to low. Candidate image pairs are then selected based on a set similarity threshold. The center point coordinates of the standard object in the sub-template images of the candidate image pairs are used to determine the center point coordinates of the target object in the image to be processed. For example, if the similarity threshold is set to 0.9, this embodiment selects 30 sub-template images that meet the standard from 180 sub-template images based on the similarity threshold. Correspondingly, when determining the center coordinates of the screw hole in the image to be processed, the center positions corresponding to the aforementioned 30 sub-template images are used. The final screw hole center coordinates determined by combining the center positions of multiple sub-template images are more reliable and accurate than those determined by a single template.

[0052] In one possible implementation, determining the coordinates of the target object's center point in the image to be processed based on the coordinates of all center points includes: Assign corresponding weights to the coordinates of all center points; The coordinates of all center points are summed with the products of their corresponding weights to obtain the weighted fusion result. The ratio of the weighted fusion result to the sum of all weights is then determined to obtain the coordinates of the center point of the target object in the image to be processed.

[0053] In this embodiment, the coordinates of the center point of the standard object are the coordinates of the center of the corresponding screw hole in the sub-template image, determined by... express,{ } indicates that a total of i template screw holes were matched. By assigning corresponding weights to the coordinates of these template screw holes, and adding the products of the coordinates of all the center points with their corresponding weights, a weighted fusion result is obtained. Based on the ratio of the weighted fusion result to the sum of all weights, the coordinates of the target object's center point in the image to be processed, i.e., the coordinates of the center position of the screw hole to be located, are determined. The corresponding formula is as follows:

[0054] In the formula, As weight, The number of screw holes in the matched template.

[0055] It should be noted that, in this embodiment, For example, based on a similarity threshold, 30 image pairs are selected from 180 image pairs, and correspondingly, 30 sub-template images are matched, i.e., n=30. =1 / 30, Substituting the center coordinates of 1 / 30 and 30 template screw holes into the above formula, the corresponding center coordinates of the screw hole to be located are calculated. By fusing multiple highly similar matching results, the final center coordinates of the screw hole to be measured are obtained, thereby improving the accuracy of the final result and achieving precise positioning.

[0056] To implement the above method, an example of this application also provides a target positioning device 800, such as... Figure 8 As shown, the device includes: The image processing module 801 is used to acquire multiple sub-images of the image to be processed and the gradient features of each sub-image. The multiple sub-images to be processed include N groups of sub-images to be processed, and each group of sub-images to be processed includes M sub-images of the same size but different resolutions. The similarity comparison module 802 is used to determine, for each sub-template image, sub-template images with the same size and resolution, to obtain multiple image pairs; The similarity comparison module 802 is also used to determine the similarity between the image pair neutron template image and the image to be processed based on the gradient features of the image pair neutron template image and the gradient features of the sub-image to be processed. The target recognition module 803 is used to select candidate image pairs whose similarity meets the threshold, and determine the position information of the target object in the image to be processed based on the position information of the standard object in the sub-template image of all candidate image pairs.

[0057] In one embodiment, the template image processing module or the image processing module 801 is further configured to determine the gradient magnitude and gradient direction of multiple image channels for each pixel in the image, and to use the median gradient magnitude as the gradient magnitude of that pixel. The gradient direction corresponding to the median gradient magnitude is taken as the gradient direction of that pixel. The target region centered on the pixel is determined, and the gradient direction of the pixel is updated using the gradient directions of other pixels in the target region. After quantization, the updated gradient direction and the gradient magnitude of the pixel are combined to form the gradient feature of the pixel. The gradient features of all pixels constitute the gradient feature of the image.

[0058] In one embodiment, the similarity comparison module 802 is further configured to determine, for each pixel of the sub-template image, the matching region corresponding to the pixel in the sub-image to be processed, wherein the matching region is the region centered on the corresponding pixel in the sub-image to be processed; The similarity between the gradient feature of the pixel and the gradient features of all pixels in the target region is compared to obtain the similarity between the pixel and the matching region. The similarity between all pixels and the matching region is summed to obtain the similarity between the sub-template image and the sub-image to be processed in the image pair.

[0059] In one embodiment, the target recognition module 803 is further configured to determine the center point of the standard object in the neutron template image of all candidate image pairs; Based on the coordinates of all center points, determine the coordinates of the center point of the target object in the image to be processed, so as to characterize the position information of the target object.

[0060] In one embodiment, the target recognition module 803 is further configured to determine the coordinates of the center point of the target object in the image to be processed based on the coordinates of all center points, including: Assign corresponding weights to the coordinates of all center points; The coordinates of all center points are summed with the products of their corresponding weights to obtain the weighted fusion result. The ratio of the weighted fusion result to the sum of all weights is then determined to obtain the coordinates of the center point of the target object in the image to be processed.

[0061] In one embodiment, the template image processing module or the image processing module 801 to be processed is further configured to determine the direction interval to which the updated gradient direction of each pixel belongs according to the quantization rule. The quantization rule is used to divide the continuous circumferential direction represented by the gradient direction into multiple direction intervals at equal intervals, and assign an interval identifier code to each direction interval. By combining the interval identifier codes corresponding to each directional interval of the pixel, an interval identifier code sequence is obtained.

[0062] By way of example, this application also provides an electronic device, including: processor; Memory used to store processor-executable instructions; The processor is used to read executable instructions from memory and execute the instructions to implement the target location method described above.

[0063] By way of example, this application also provides a computer-readable storage medium storing a computer program for performing the target localization method described above.

[0064] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0065] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0066] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0067] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as a target localization method. For example, in some embodiments, a target localization method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of a target localization method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a target localization method by any other suitable means (e.g., by means of firmware).

[0068] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0070] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0071] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0072] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0073] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0074] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0075] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0076] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A target localization method, characterized in that, The method includes: The process involves acquiring multiple sub-images of the image to be processed and the gradient features of each sub-image. The multiple sub-images to be processed include N groups of sub-images to be processed, and each group of sub-images to be processed includes M sub-images of the same size but different resolutions. For each sub-image to be processed, determine a sub-template image with the same size and resolution to obtain multiple image pairs; The similarity between the neutron template image and the sub-image to be processed is determined based on the gradient features of the neutron template image and the gradient features of the sub-image to be processed. Candidate image pairs whose similarity meets the threshold are selected, and the position information of the target object in the image to be processed is determined based on the position information of the standard object in the sub-template image of all candidate image pairs.

2. The target localization method according to claim 1, characterized in that, Obtaining the gradient features of the sub-template image or the gradient features of the sub-image to be processed includes: Determine the gradient magnitude and gradient direction of multiple image channels for each pixel in the image, and use the median gradient magnitude as the gradient magnitude of that pixel; The gradient direction corresponding to the median gradient magnitude is taken as the gradient direction of the pixel. A target region centered on the pixel is determined, and the gradient direction of the pixel is updated using the gradient directions of other pixels within the target region. After quantization, the updated gradient direction is combined with the gradient magnitude of the pixel to form the gradient feature of the pixel, and the gradient features of all pixels constitute the gradient feature of the image.

3. The target localization method according to claim 1, characterized in that, Determining the similarity between the neutron template image and the sub-image to be processed based on the gradient features of the neutron template image and the gradient features of the sub-image to be processed includes: For each pixel in the sub-template image, determine the matching region corresponding to that pixel in the sub-image to be processed. The matching region is the region centered on the corresponding pixel in the sub-image to be processed. The similarity between the gradient feature of the pixel and the gradient features of all pixels in the matching region is obtained by comparing the similarity between the pixel and the matching region. The similarity between all pixels and the matching region is summed to obtain the similarity between the sub-template image and the sub-image to be processed in the image pair.

4. The target localization method according to claim 1, characterized in that, The step of determining the position information of the target object in the image to be processed based on the position information of the standard objects in the neutron template image of all candidate images includes: Determine the center point of the standard object in the neutron template image of all candidate image pairs; Based on the coordinates of all center points, the coordinates of the center point of the target object in the image to be processed are determined to characterize the position information of the target object.

5. The target localization method according to claim 4, characterized in that, Determining the coordinates of the target object's center point in the image to be processed based on the coordinates of all center points includes: Assign corresponding weights to the coordinates of all center points; The coordinates of all center points are multiplied by their corresponding weights to obtain a weighted fusion result. The ratio of the weighted fusion result to the sum of all weights is then determined to obtain the coordinates of the center point of the target object in the image to be processed.

6. The target localization method according to claim 2, characterized in that, The quantization process for the updated gradient direction includes: For each pixel, the direction interval to which the updated gradient direction of the pixel belongs is determined according to the quantization rule. The quantization rule is used to divide the continuous circumferential direction represented by the gradient direction into multiple direction intervals at equal intervals, and assign an interval identifier code to each direction interval. By combining the interval identifier codes corresponding to each directional interval of the pixel, an interval identifier code sequence is obtained.

7. A target positioning device, characterized in that, The device includes: The image processing module is used to acquire multiple sub-images of the image to be processed and the gradient features of each sub-image. The multiple sub-images to be processed include N sub-image groups, and each sub-image group includes M sub-images of the same size but different resolutions. The similarity comparison module is used to identify sub-template images with the same size and resolution for each sub-template image, thus obtaining multiple image pairs. The similarity comparison module is further configured to determine the similarity between the image pair neutron template image and the image to be processed based on the gradient features of the image pair neutron template image and the gradient features of the sub-image to be processed. The target recognition module is used to select candidate image pairs whose similarity meets the threshold, and determine the position information of the target object in the image to be processed based on the position information of the standard object in the sub-template image of all candidate image pairs.

8. The target positioning device according to claim 7, characterized in that, The similarity comparison module is also used to determine the matching region corresponding to each pixel in the sub-image to be processed for each pixel in the sub-template image, wherein the matching region is the region centered on the corresponding pixel in the sub-image to be processed. The similarity between the gradient feature of the pixel and the gradient features of all pixels in the target region is compared to obtain the similarity between the pixel and the matching region. The similarity between all pixels and the matching region is summed to obtain the similarity between the sub-template image and the sub-image to be processed in the image pair.

9. An electronic device, characterized in that, include: At least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform claim 1. The target localization method as described in any one of the 6.

10. A computer-readable storage medium storing a computer program for performing the target localization method according to any one of claims 1-6.

Images