Method and apparatus for centering cross-shaped targets
By extracting a coarse localization image from the initial grayscale image and determining the center of the crosshair target using gradient calculation and grid division, the problems of insufficient environmental adaptability and accuracy in existing technologies are solved, and high-precision localization in complex environments is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN GUIDE INFRARED CO LTD
- Filing Date
- 2025-06-09
- Publication Date
- 2026-07-17
AI Technical Summary
Existing crosshair target centering technology has shortcomings in terms of environmental adaptability and positioning accuracy, especially in complex environments where it is difficult to achieve high-precision positioning.
By extracting a coarse localization image from the initial grayscale image, the target row and column matrix is determined using gradient calculation and grid division. The center score of the candidate rectangular blocks is calculated using preset rules, and the rectangular block with the highest score and greater than the threshold is selected as the center of the cross target.
It can quickly and accurately locate the center of a crosshair target in visible light, infrared, and low-light environments, effectively resisting environmental interference and improving the accuracy and adaptability of positioning.
Smart Images
Figure CN120747213B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image detection technology, specifically to a method and apparatus for locating the center of a crosshatch target. Background Technology
[0002] In the fields of machine vision and image detection, crosshair target center localization technology plays a crucial role in scenarios such as optical instrument calibration, industrial automation measurement, and UAV navigation. Current mainstream technologies mainly fall into two categories: First, converting the grayscale image into a binary image, searching for quadrilateral contours that meet the error threshold within the binary image, and locating the crosshair within the approximately rectangular contour; Second, binarizing the grayscale image using an iterative method, combining edge extraction and Hough transform to extract straight line features, and obtaining the crosshair target center information through straight line grouping fitting, orthogonal calculation, and intersection point clustering.
[0003] However, existing technologies have significant limitations. On the one hand, traditional binarization methods heavily rely on threshold settings. When image brightness distribution is uneven or there is interference information brighter than the target, it is difficult to accurately set the threshold, leading to distorted binarization results. In low-quality images, background noise easily mixes into the segmentation results, severely affecting the accuracy of target center localization. On the other hand, line detection and intersection calculation methods based on Hough transform require repeated adjustments to parameters such as thresholds to adapt to different image scenarios, resulting in insufficient program generalization ability. Especially in long-distance observation scenarios, image pixel changes are drastic, boundaries are blurred, line slopes fluctuate frequently, and sometimes lines cannot be effectively extracted, greatly restricting the reliability and practicality of target center localization technology. Therefore, there is an urgent need for a high-precision target center localization method that can overcome the above defects and adapt to complex environments. Summary of the Invention
[0004] This application provides a method and apparatus for locating the center of a crosshatch target, which can solve the technical problems of poor environmental adaptability and low positioning accuracy in the prior art.
[0005] In a first aspect, embodiments of this application provide a method for locating the center of a cross-shaped target, the method comprising: A coarse localization image is extracted based on the initial grayscale image and the corresponding candidate regions; A first intermediate image is generated based on a coarse localization image, wherein the operation of generating the first intermediate image includes gradient calculation of the coarse localization image; The target row matrix and target column matrix are determined from the first intermediate image. The pixels at the corresponding positions in the coarse localization image are used as marker points. The coarse localization image is divided into regions to obtain multiple candidate rectangular blocks. Each row matrix has an equal number of pixel rows, and each column matrix has an equal number of pixel columns. The target row matrix is a matrix of multiple rows with the largest weighted sum of pixel values, and the target column matrix is a matrix of multiple columns with the largest weighted sum of pixel values. The candidate rectangular blocks include a main body composed of unmarked points. The center score of each candidate rectangular block is calculated according to preset rules. The preset rules include: the smaller the difference in average pixel value between the current candidate rectangular block and its horizontal and vertical adjacent candidate rectangular blocks, and the larger the difference in average pixel value with its diagonal adjacent candidate rectangular blocks, the higher the center score of the current candidate rectangular block. The candidate rectangular block with the highest center score that is greater than the score threshold is taken as the target rectangular block, and the center of the target rectangular block is taken as the center of the cross target corresponding to the initial grayscale image.
[0006] Furthermore, in one embodiment, the candidate rectangular block further includes a border portion composed of marker points; For each candidate rectangle, its border portion completely overlaps with the border portions of its horizontally and vertically adjacent candidate rectangle portions.
[0007] Furthermore, in one embodiment, the step of generating the first intermediate image based on the coarse localization image includes: Gradient calculation and filtering are performed on the coarse localization image to obtain the second intermediate image; Gradient enhancement is performed on the second intermediate image to obtain the first intermediate image.
[0008] Further, in one embodiment, the step of performing gradient enhancement on the second intermediate image to obtain the first intermediate image includes: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the third intermediate image.
[0009] Further, in one embodiment, the step of performing gradient enhancement on the second intermediate image to obtain the first intermediate image includes: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. Iterate through each pixel in the third intermediate image. If the number of foreground pixels in the current pixel and its horizontal, vertical, and diagonal neighboring pixels is greater than or equal to a preset threshold, then set the pixel at the corresponding position in the fourth intermediate image as a foreground pixel; otherwise, set the pixel at the corresponding position in the fourth intermediate image as a background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the fourth intermediate image.
[0010] Furthermore, in one embodiment, the cross-shaped target center localization method further includes: Iterate through each initial grayscale image in the video stream sequence and perform the following operations: Add the current frame to the latest detection group; If a target rectangle exists in the current frame, the intersection-union ratio of the target rectangle in the current frame with the target rectangle in each frame of each detection group is calculated. If there is an intersection-over-union ratio (IoU) greater than zero, the current frame is added to the detection group with the largest IoU average; otherwise, the current frame is added to the newly created detection group. If the number of frames in the latest detection group is equal to the frame number threshold, then the detection group with the most frames is selected as the best detection group. The center of the crosshair target corresponding to the latest detection group is determined based on the center of the target rectangle block corresponding to at least one frame in the best detection group. A new detection group is created, and all detection groups are deleted.
[0011] Further, in one embodiment, the step of determining the center of the crosshair target corresponding to the latest detection group based on the center of the target rectangular block corresponding to at least one frame in the optimal detection group includes: The center of the target rectangle in the latest frame of the best detection group is taken as the center of the cross-shaped target in the latest detection group.
[0012] Furthermore, in one embodiment, the method further includes the following step before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If the initial grayscale image does not have a corresponding candidate region, then the initial grayscale image is downsampled, filtered, and gradient calculated to obtain the fifth intermediate image; The target region is obtained by extracting local maxima in the fifth intermediate image. The target region is expanded by image dilation. The smallest bounding rectangle of the target region is used as the candidate region corresponding to the initial grayscale image.
[0013] Further, in one embodiment, the step of downsampling, filtering, and gradient calculation of the initial grayscale image to obtain a fifth intermediate image if no corresponding candidate region exists in the initial grayscale image includes: If the initial grayscale image does not have a corresponding candidate region and the initial grayscale image is an inner field image, then the initial grayscale image is downsampled, filtered and gradient calculated to obtain the fifth intermediate image; The method further includes the following steps before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If no candidate region exists and the initial grayscale image is an external image, the user is prompted to set a candidate region corresponding to the initial grayscale image.
[0014] Secondly, embodiments of this application also provide a cross-shaped target center positioning device, the cross-shaped target center positioning device comprising: The image cropping module is used to crop a coarse localization image based on the initial grayscale image and the corresponding candidate region; An image conversion module is used to generate a first intermediate image based on a coarse localization image, wherein the operation of generating the first intermediate image includes performing gradient calculation on the coarse localization image; The grid partitioning module is used to determine the target row matrix and target column matrix from the first intermediate image. Pixels at corresponding positions in the coarse localization image are used as marker points. The coarse localization image is divided into regions to obtain multiple candidate rectangular blocks. Each row matrix has an equal number of pixel rows, and each column matrix has an equal number of pixel columns. The target row matrix is a set of multiple row matrices with the largest weighted sum of pixel values, and the target column matrix is a set of multiple column matrices with the largest weighted sum of pixel values. The candidate rectangular blocks include a main body composed of unmarked points. The score calculation module is used to calculate the center score of each candidate rectangular block according to preset rules. The preset rules include: the smaller the difference in average pixel value between the current candidate rectangular block and its horizontal and vertical adjacent candidate rectangular blocks, and the larger the difference in average pixel value with its diagonal adjacent candidate rectangular blocks, the higher the center score of the current candidate rectangular block. The center localization module is used to select the candidate rectangular block with the highest center score that is greater than the score threshold as the target rectangular block, and to use the center of the target rectangular block as the center of the cross target corresponding to the initial grayscale image.
[0015] In this application, a coarse localization image is extracted from the initial grayscale image. Operations including gradient calculation are used to enhance the image features of the crosshair target. Row and column matrices with high pixel weighting values are selected as candidates for the crosshair target boundary. The coarse localization image is then divided into a grid. Each candidate rectangular block is compared with its corresponding eight-neighbor candidate rectangular blocks. The comparison results are scored according to the image patterns of the crosshair target. The center of the candidate rectangular block with the highest score exceeding a threshold is taken as the center of the crosshair target corresponding to the initial grayscale image. This application effectively resists environmental interference and can quickly and accurately locate the center of the crosshair target in visible light, infrared, and low-light scenarios, demonstrating high application value. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating a method for locating the center of a crosshatch target in one embodiment of this application. Figure 2 This is a schematic diagram illustrating the operation of marking pixels in a coarse positioning image according to an embodiment of this application; Figure 3 This is a schematic diagram illustrating the region division of a coarse positioning image in one embodiment of this application; Figure 4 This is a schematic diagram illustrating the operation of generating a first intermediate image in one embodiment of this application; Figure 5 This is a schematic diagram of the process for processing a video stream sequence in one embodiment of this application; Figure 6 This is a schematic diagram illustrating the operation of automatically detecting and generating candidate regions in one embodiment of this application; Figure 7 This is a schematic diagram illustrating the effect of automatic candidate region generation and center localization of an indoor cross-shaped target under different test scenarios in this application. Figure 8 This is a schematic diagram illustrating the effect of centering a cross-shaped target in a given candidate region under different experimental scenarios in this application. Figure 9 This is a schematic diagram of the functional modules of the cross-shaped target center positioning device in one embodiment of this application. Detailed Implementation
[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0019] In a first aspect, embodiments of this application provide a method for locating the center of a cross-shaped target.
[0020] Figure 1 A flowchart illustrating a method for locating the center of a crosshatch target in one embodiment of this application is shown.
[0021] Reference Figure 1 In one embodiment, the method for locating the center of a crosshair target includes the following steps: S1. Extract a coarse localization image based on the initial grayscale image and the corresponding candidate regions.
[0022] It is understandable that performing fine positioning of the crosshair target center directly across the entire map would increase the time consumption and cause a loss of accuracy. Therefore, this embodiment first completes the coarse positioning of the crosshair target.
[0023] Optionally, the candidate region corresponding to the initial grayscale image can be automatically detected and generated by the program or manually set by the user; this embodiment does not limit this.
[0024] It should be noted that if the original captured image is not a grayscale image (single-channel image), it needs to be converted to a grayscale image for processing.
[0025] S2. Generate a first intermediate image based on the coarse localization image, wherein the operation of generating the first intermediate image includes gradient calculation of the coarse localization image.
[0026] Specifically, gradient calculation is a commonly used edge detection method. Performing operations including gradient calculation on a coarsely located image can extract the edge parts in the coarsely located image.
[0027] Alternatively, other methods can be used to assist in generating the first intermediate image in order to improve the edge extraction effect.
[0028] S3. Determine the target row matrix and target column matrix from the second intermediate image. Use the pixels at the corresponding positions in the coarse localization image as marker points. Divide the coarse localization image into regions to obtain multiple candidate rectangular blocks. Each row matrix has an equal number of pixel rows, and each column matrix has an equal number of pixel columns. The target row matrix is a set of multiple row matrices with the largest weighted sum of pixel values, and the target column matrix is a set of multiple column matrices with the largest weighted sum of pixel values. The candidate rectangular blocks include the main body composed of unmarked points.
[0029] In this embodiment, considering that the boundary of the cross target has obvious gradient information in the image, in the first intermediate image, the sum of the pixel values of the row and column where the boundary of the cross target is located should be significantly higher than that of other rows and columns. Therefore, by filtering the rows and columns of the second intermediate image by the sum of the pixel values, candidates for the boundary of the cross target can be found.
[0030] Furthermore, considering that the crosshair target may rotate relative to the row and column directions of the image, using each row and column individually as detection units would result in significant detection errors when dealing with a rotating crosshair target. Therefore, the detection units in this embodiment are defined as row and column matrices, meaning that pixel values can be summed for a row of pixels or multiple consecutive rows of pixels, or a column of pixels or multiple consecutive columns of pixels, thereby adapting to rotating crosshair targets.
[0031] Note that in row and column matrices, each row and column can have different weights or the same weight. For example, when the row matrix consists of three consecutive rows of pixels, the middle row has a higher weight, while the top and bottom rows have lower weights.
[0032] As an optional implementation, the pixel values of all row matrices and all column matrices are weighted and sorted from high to low, and the row matrices and column matrices that are sorted within a set range are selected as the target row matrix and target column matrix.
[0033] As another optional implementation, the average or median value of the weighted sum of pixel values of all row matrices and all column matrices is calculated respectively, and the row matrices and column matrices with values higher than the average or median value are used as the target row matrices and target column matrices.
[0034] Figure 2 This illustration shows a schematic diagram of the operation of marking pixels in a coarse positioning image according to an embodiment of this application.
[0035] Reference Figure 2 The coarse localization image shown in sub-image (a) undergoes operations including gradient calculation to obtain the first intermediate image shown in sub-image (b). The determined target row matrix and target column matrix are referenced in sub-image (c) and are in a grid-like structure. The image is then transferred to sub-image (a) according to the grid in sub-image (c), resulting in the image shown in sub-image (d). It can be seen that the grid divides sub-image (c) into multiple rectangular blocks.
[0036] Specifically, the candidate rectangular block required in this embodiment includes a main body composed of unmarked points.
[0037] For example, Figure 2 Each rectangular block in the subgraph (d) that does not contain green pixels belongs to a corresponding candidate rectangular block.
[0038] S4. Calculate the center score of each candidate rectangular block according to preset rules. The preset rules include: the smaller the difference in average pixel value between the current candidate rectangular block and its horizontal and vertical adjacent candidate rectangular blocks, and the larger the difference in average pixel value between the current candidate rectangular block and its diagonal adjacent candidate rectangular blocks, the higher the center score of the current candidate rectangular block.
[0039] In this embodiment, the average pixel value of each candidate rectangular block is compared with that of its neighboring candidate rectangular blocks in the horizontal, vertical and diagonal directions. A score is given based on the comparison results, and the probability that each candidate rectangular block contains the center of the cross target is quantified by the score.
[0040] Continue to refer to Figure 2In subimage (d), the candidate rectangle containing the center of the crosshair target is the candidate rectangle in the fourth row and fifth column. This candidate rectangle is dark overall, and the four candidate rectangles above, below, left, and right of it are also dark overall, with average pixel values close to the center. The four candidate rectangles diagonally opposite it are bright overall, with average pixel values far from the center.
[0041] Figure 2 The cross-shaped target in the image is a dark cross-shaped target with a dark target and a bright background. The bright cross-shaped target is the opposite, with a bright target and a dark background. The difference in average pixel values follows the same pattern. Therefore, this embodiment is compatible with both dark and bright cross-shaped targets.
[0042] S5. Select the candidate rectangular block with the highest center score that is greater than the score threshold as the target rectangular block, and use the center of the target rectangular block as the center of the cross target corresponding to the initial grayscale image.
[0043] In this embodiment, considering that the cross-shaped target may not exist in the initial grayscale image, or the quality of the initial grayscale image may be too low to accurately identify the center of the cross-shaped target, when selecting the target rectangular block, not only the relative size of the center fraction but also the absolute size of the center fraction should be considered to ensure the accuracy of the cross-shaped target center.
[0044] Therefore, in this embodiment, a coarse localization image is extracted from the initial grayscale image. Operations including gradient calculation are used to enhance the image features of the crosshair target. Row and column matrices with higher pixel weights are selected as candidates for the crosshair target boundary. The coarse localization image is then divided into a grid. Each candidate rectangular block is compared with its corresponding eight-neighbor candidate rectangular blocks. The comparison results are scored according to the image patterns of the crosshair target. The center of the candidate rectangular block with the highest score exceeding a threshold is taken as the center of the crosshair target corresponding to the initial grayscale image. This embodiment effectively resists environmental interference and can quickly and accurately locate the center of the crosshair target in visible light, infrared, and low-light scenarios, demonstrating high application value.
[0045] Furthermore, in one embodiment, the candidate rectangular block further includes a border portion composed of marker points; For each candidate rectangle, its border portion completely overlaps with the border portions of its horizontally and vertically adjacent candidate rectangle portions.
[0046] It is understandable that if the candidate rectangular block only includes the main body composed of non-marked points and excludes the marked point part, some information will be lost. This problem of information loss is particularly obvious when the number of pixel rows in the row matrix is large and the number of pixel columns in the column matrix is large, which can easily lead to false detection or false negative detection of the target rectangular block.
[0047] In this embodiment, the marker points are used as the border of the candidate rectangular blocks, and the adjacent candidate rectangular blocks are controlled to overlap to the greatest extent, thereby avoiding the loss of effective information at the marker points and further improving the accuracy of the cross-shaped target center positioning.
[0048] Figure 3 This is a schematic diagram of region division of a coarse positioning image in one embodiment of this application.
[0049] Reference Figure 3 Assuming both the coarse localization image and the first intermediate image are 100 pixels in width and height, with a row matrix of 6 consecutive rows of pixels and a column matrix of 9 consecutive columns of pixels, when detecting the target row matrix and target column matrix in the first intermediate image, it is necessary to calculate the weighted sum of pixel values in rows 1-6, 2-7, ..., 95-100, and the weighted sum of pixel values in columns 1-9, 2-10, ..., 92-100, respectively. Assuming the final target row matrix determined from the first intermediate image is rows 33-38 and 60-65, and the target column matrix is columns 33-41 and 60-68, 9 candidate rectangular blocks are divided in the coarse localization image.
[0050] According to the division method in this embodiment, the upper and lower boundaries of rectangles 1, 2, and 3 are in row 1 and row 38, respectively; the upper and lower boundaries of rectangles 4, 5, and 6 are in row 33 and row 65, respectively; the upper and lower boundaries of rectangles 7, 8, and 9 are in row 60 and row 100, respectively; the left and right boundaries of rectangles 1, 4, and 7 are in column 1 and column 41, respectively; the left and right boundaries of rectangles 2, 5, and 8 are in column 33 and column 68, respectively; and the left and right boundaries of rectangles 3, 6, and 9 are in column 60 and column 100, respectively.
[0051] Furthermore, in one embodiment, the step of generating the first intermediate image based on the coarse localization image includes: Gradient calculation and filtering are performed on the coarse localization image to obtain the second intermediate image; Gradient enhancement is performed on the second intermediate image to obtain the first intermediate image.
[0052] In this embodiment, filtering and gradient enhancement are used to improve the edge extraction effect based on gradient calculation.
[0053] Further, in one embodiment, the step of performing gradient enhancement on the second intermediate image to obtain the first intermediate image includes: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the third intermediate image.
[0054] In this embodiment, based on the gradient strength relationship between each pixel and its four neighboring pixels, the second intermediate image is binarized to obtain the third intermediate image. The foreground pixels of the third intermediate image represent the positions with stronger local gradients, and the background pixels represent the positions with weaker local gradients. In the first intermediate image obtained by multiplying the second and third intermediate images, the pixels of the second intermediate image are retained at the corresponding positions in the foreground region of the third intermediate image, and the corresponding positions in the background region of the third intermediate image are black backgrounds, thus achieving the purpose of gradient enhancement.
[0055] Further, in one embodiment, the step of performing gradient enhancement on the second intermediate image to obtain the first intermediate image includes: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. Iterate through each pixel in the third intermediate image. If the number of foreground pixels in the current pixel and its horizontal, vertical, and diagonal neighboring pixels is greater than or equal to a preset threshold, then set the pixel at the corresponding position in the fourth intermediate image as a foreground pixel; otherwise, set the pixel at the corresponding position in the fourth intermediate image as a background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the fourth intermediate image.
[0056] This embodiment, based on the previous embodiment, removes isolated points from the third intermediate image to obtain a fourth intermediate image, based on the number of foreground pixels in each pixel and its eight neighboring pixels. In a crosshair target image, isolated foreground pixels are generally considered noise. The first intermediate image, obtained by multiplying the fourth intermediate image (after removing isolated points) with the second intermediate image, has higher accuracy in determining the target row matrix and target column matrix due to further noise removal.
[0057] Figure 4 A schematic diagram illustrating the operation of generating a first intermediate image in one embodiment of this application is shown.
[0058] Reference Figure 4The coarse localization image shown in sub-image (a) is processed through gradient calculation and filtering to obtain the second intermediate image shown in sub-image (b). The second intermediate image is then binarized to obtain the third intermediate image shown in sub-image (c). The third intermediate image contains some isolated points, which are removed to obtain the fourth intermediate image shown in sub-image (d). The second and fourth intermediate images are then multiplied to obtain the first intermediate image shown in sub-image (e). Comparing sub-images (b) and (e), it can be seen that the edge contours in sub-image (e) are clearer.
[0059] Figure 5 A flowchart illustrating the processing of a video stream sequence is shown in one embodiment of this application.
[0060] Reference Figure 5 Furthermore, in one embodiment, the cross-shaped target center localization method further includes: Iterate through each initial grayscale image in the video stream sequence and perform the following operations: Add the current frame to the latest detection group; If a target rectangle exists in the current frame, the intersection-union ratio of the target rectangle in the current frame with the target rectangle in each frame of each detection group is calculated. If there is an intersection-over-union ratio (IoU) greater than zero, the current frame is added to the detection group with the largest IoU average; otherwise, the current frame is added to the newly created detection group. If the number of frames in the latest detection group is equal to the frame number threshold, then the detection group with the most frames is selected as the best detection group. The center of the crosshair target corresponding to the latest detection group is determined based on the center of the target rectangle block corresponding to at least one frame in the best detection group. A new detection group is created, and all detection groups are deleted.
[0061] The foregoing embodiments described how to locate the center of a crosshair target in a single frame of initial grayscale image. This embodiment further describes how to locate the center of a crosshair target in multiple consecutive frames of initial grayscale image.
[0062] In this embodiment, video frames are divided into different detection groups based on a frame count threshold. Within each detection group, the video frames are further divided into multiple detection subgroups based on the intersection-over-union (IoU) ratio of the target rectangles corresponding to different frames. The detection subgroup with the most frames is selected as the optimal detection subgroup. The center of the crosshair target corresponding to the detection group is determined based on the center of the target rectangle corresponding to at least one frame in the optimal detection subgroup. This embodiment enables the output of a more accurate and reliable crosshair target center using the video stream sequence.
[0063] Furthermore, the step of determining the center of the crosshair target corresponding to the latest detection group based on the center of the target rectangular block corresponding to at least one frame in the best detection group includes: The center of the target rectangle in the latest frame of the best detection group is taken as the center of the cross-shaped target in the latest detection group.
[0064] In this embodiment, in the best detection group, the center of the crosshair target corresponding to the latest frame in the time series is considered to be the most accurate, and it is output as the result of the detection group.
[0065] Of course, in other embodiments, other processing methods can also be used, such as taking the average result of the center of the target rectangle block corresponding to each frame in the best detection group as the center of the cross target corresponding to the latest detection group.
[0066] Furthermore, in one embodiment, the method further includes the following step before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If the initial grayscale image does not have a corresponding candidate region, then the initial grayscale image is downsampled, filtered, and gradient calculated to obtain the fifth intermediate image; The target region is obtained by extracting local maxima in the fifth intermediate image. The target region is expanded by image dilation. The smallest bounding rectangle of the target region is used as the candidate region corresponding to the initial grayscale image.
[0067] In this embodiment, a method for automatically detecting and generating candidate regions is provided. Since operations are required across the entire image, downsampling the image can improve the processing speed while meeting the requirements of coarse localization, thus ensuring the real-time output.
[0068] Figure 6 A schematic diagram of the operation of automatically detecting and generating candidate regions is shown in one embodiment of this application.
[0069] Reference Figure 6 The initial grayscale image shown in sub-image (a) is downsampled, filtered, and gradient calculated to obtain the fifth intermediate image shown in sub-image (b). Local maxima are extracted from the fifth intermediate image and image dilation is performed. The target region is the red region in sub-image (c). The minimum bounding rectangle of the target region is drawn, and the candidate region is the white box in sub-image (c). Finally, the coarse positioning image shown in sub-image (d) is extracted.
[0070] Further, in one embodiment, the step of downsampling, filtering, and gradient calculation of the initial grayscale image to obtain a fifth intermediate image if no corresponding candidate region exists in the initial grayscale image includes: If the initial grayscale image does not have a corresponding candidate region and the initial grayscale image is an inner field image, then the initial grayscale image is downsampled, filtered and gradient calculated to obtain the fifth intermediate image; The method further includes the following steps before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If no candidate region exists and the initial grayscale image is an external image, the user is prompted to set a candidate region corresponding to the initial grayscale image.
[0071] Specifically, in the application environment of crosshair target center point positioning, crosshair target positioning is divided into indoor and outdoor fields. Typical indoor application scenarios include axis alignment of optoelectronic pods, while typical outdoor application scenarios include auxiliary locking of optoelectronic equipment. In the indoor environment, the approximate position of the crosshair target is usually not given, while in the outdoor environment, the approximate position of the crosshair target, i.e., the candidate area, is usually given manually.
[0072] In this embodiment, considering the complexity of the background in the outdoor environment, the reliability of automatically detected candidate regions is not high. Therefore, automatic detection and generation of candidate regions are only performed for indoor images. For outdoor images, if no given candidate region is obtained, the user will be prompted to set it manually.
[0073] Figure 7 This is a schematic diagram illustrating the effect of automatic candidate region generation and center positioning of the indoor cross-shaped target in different test scenarios according to this application.
[0074] Reference Figure 7 In terms of locating the center point of the cross-shaped target in the field, the method proposed in this application can effectively deal with input images in different scenarios, including uneven brightness, various background types, weak target characteristics, variable target size, target rotation, etc., and accurately locate the center position of the cross-shaped target.
[0075] Figure 8 This is a schematic diagram illustrating the effect of centering a cross-shaped target in a given candidate region under different experimental scenarios.
[0076] Reference Figure 8 The upper part represents the positioning effect on the infrared image, and the lower part represents the positioning effect on the visible light image. Different candidate regions are given for each instance to complete the center positioning of the crosshair target. In the field environment, the center positioning of the crosshair target faces difficulties such as complex scene, imaging loss, target rotation, coexistence of bright and dark areas, and uncertain candidate positions. The method proposed in this application can accurately locate the center position of the crosshair target.
[0077] Secondly, embodiments of this application also provide a cross-shaped target center positioning device.
[0078] Figure 9 A schematic diagram of the functional modules of a cross-shaped target center positioning device in one embodiment of this application is shown.
[0079] Reference Figure 9In one embodiment, the cross-shaped target center positioning device includes: Image cropping module 10 is used to crop a coarse localization image based on the initial grayscale image and the corresponding candidate region; Image conversion module 20 is used to generate a first intermediate image based on a coarse localization image, wherein the operation of generating the first intermediate image includes performing gradient calculation on the coarse localization image; The grid division module 30 is used to determine the target row matrix and target column matrix from the first intermediate image, use the pixels at corresponding positions in the coarse positioning image as marker points, divide the coarse positioning image into regions to obtain multiple candidate rectangular blocks, wherein each row matrix has an equal number of pixel rows, each column matrix has an equal number of pixel columns, the target row matrix is a multiple row matrix with the largest weighted sum of pixel values, the target column matrix is a multiple column matrix with the largest weighted sum of pixel values, and the candidate rectangular blocks include a main body composed of unmarked points; The score calculation module 40 is used to calculate the center score of each candidate rectangular block according to preset rules. The preset rules include: the smaller the difference in average pixel value between the current candidate rectangular block and its horizontal and vertical adjacent candidate rectangular blocks, and the larger the difference in average pixel value with its diagonal adjacent candidate rectangular blocks, the higher the center score of the current candidate rectangular block. The center localization module 50 is used to select the candidate rectangular block with the highest center score that is greater than the score threshold as the target rectangular block, and to use the center of the target rectangular block as the center of the cross target corresponding to the initial grayscale image.
[0080] Furthermore, in one embodiment, the candidate rectangular block further includes a border portion composed of marker points; For each candidate rectangle, its border portion completely overlaps with the border portions of its horizontally and vertically adjacent candidate rectangle portions.
[0081] Furthermore, in one embodiment, the image conversion module 20 is used for: Gradient calculation and filtering are performed on the coarse localization image to obtain the second intermediate image; Gradient enhancement is performed on the second intermediate image to obtain the first intermediate image.
[0082] Furthermore, in one embodiment, the image conversion module 20 is used for: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the third intermediate image.
[0083] Furthermore, in one embodiment, the image conversion module 20 is used for: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. Iterate through each pixel in the third intermediate image. If the number of foreground pixels in the current pixel and its horizontal, vertical, and diagonal neighboring pixels is greater than or equal to a preset threshold, then set the pixel at the corresponding position in the fourth intermediate image as a foreground pixel; otherwise, set the pixel at the corresponding position in the fourth intermediate image as a background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the fourth intermediate image.
[0084] Furthermore, in one embodiment, the cross-shaped target center positioning device further includes a voting module for: Iterate through each initial grayscale image in the video stream sequence and perform the following operations: Add the current frame to the latest detection group; If a target rectangle exists in the current frame, the intersection-union ratio of the target rectangle in the current frame with the target rectangle in each frame of each detection group is calculated. If there is an intersection-over-union ratio (IoU) greater than zero, the current frame is added to the detection group with the largest IoU average; otherwise, the current frame is added to the newly created detection group. If the number of frames in the latest detection group is equal to the frame number threshold, then the detection group with the most frames is selected as the best detection group. The center of the crosshair target corresponding to the latest detection group is determined based on the center of the target rectangle block corresponding to at least one frame in the best detection group. A new detection group is created, and all detection groups are deleted.
[0085] Furthermore, in one embodiment, the voting module is used for: The center of the target rectangle in the latest frame of the best detection group is taken as the center of the cross-shaped target in the latest detection group.
[0086] Furthermore, in one embodiment, the cross-shaped target center positioning device further includes a region detection module, used for: If the initial grayscale image does not have a corresponding candidate region, then the initial grayscale image is downsampled, filtered, and gradient calculated to obtain the fifth intermediate image; The target region is obtained by extracting local maxima in the fifth intermediate image. The target region is expanded by image dilation. The smallest bounding rectangle of the target region is used as the candidate region corresponding to the initial grayscale image.
[0087] Furthermore, in one embodiment, the region detection module is used for: If the initial grayscale image does not have a corresponding candidate region and the initial grayscale image is an inner field image, then the initial grayscale image is downsampled, filtered and gradient calculated to obtain the fifth intermediate image; The method further includes the following steps before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If no candidate region exists and the initial grayscale image is an external image, the user is prompted to set a candidate region corresponding to the initial grayscale image.
[0088] The functions of each module in the above-mentioned cross-target center positioning device correspond to the steps in the above-mentioned cross-target center positioning method embodiment, and their functions and implementation processes will not be described in detail here.
[0089] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0090] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0091] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0092] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0093] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0095] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A method for locating the center of a cross-shaped target, characterized in that, The method for locating the center of the crosshair target includes: A coarse localization image is extracted based on the initial grayscale image and the corresponding candidate regions; A first intermediate image is generated based on a coarse localization image, wherein the operation of generating the first intermediate image includes gradient calculation of the coarse localization image; The target row matrix and target column matrix are determined from the first intermediate image. The pixels at the corresponding positions in the coarse localization image are used as marker points. The coarse localization image is divided into regions to obtain multiple candidate rectangular blocks. Each row matrix has an equal number of pixel rows, and each column matrix has an equal number of pixel columns. The target row matrix is a matrix of multiple rows with the largest weighted sum of pixel values, and the target column matrix is a matrix of multiple columns with the largest weighted sum of pixel values. The candidate rectangular blocks include a main body composed of unmarked points. The center score of each candidate rectangular block is calculated according to preset rules. The preset rules include: the smaller the difference in average pixel value between the current candidate rectangular block and its horizontal and vertical adjacent candidate rectangular blocks, and the larger the difference in average pixel value with its diagonal adjacent candidate rectangular blocks, the higher the center score of the current candidate rectangular block. The candidate rectangular block with the highest center score that is greater than the score threshold is taken as the target rectangular block, and the center of the target rectangular block is taken as the center of the cross target corresponding to the initial grayscale image.
2. The method for locating the center of a cross-shaped target as described in claim 1, characterized in that, The candidate rectangular block also includes a border portion composed of marker points; For each candidate rectangle, its border portion completely overlaps with the border portions of its horizontally and vertically adjacent candidate rectangle portions.
3. The method for locating the center of a cross-shaped target as described in claim 1, characterized in that, The step of generating the first intermediate image based on the coarse localization image includes: Gradient calculation and filtering are performed on the coarse localization image to obtain the second intermediate image; Gradient enhancement is performed on the second intermediate image to obtain the first intermediate image.
4. The method for locating the center of a cross-shaped target as described in claim 3, characterized in that, The step of performing gradient enhancement on the second intermediate image to obtain the first intermediate image includes: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the third intermediate image.
5. The method for locating the center of a cross-shaped target as described in claim 3, characterized in that, The step of performing gradient enhancement on the second intermediate image to obtain the first intermediate image includes: Iterate through each pixel in the second intermediate image. If the gradient of the current pixel is stronger than the gradient of its horizontal and vertical neighboring pixels, set the pixel at the corresponding position in the third intermediate image as the foreground pixel; otherwise, set the pixel at the corresponding position in the third intermediate image as the background pixel. Iterate through each pixel in the third intermediate image. If the number of foreground pixels in the current pixel and its horizontal, vertical, and diagonal neighboring pixels is greater than or equal to a preset threshold, then set the pixel at the corresponding position in the fourth intermediate image as a foreground pixel; otherwise, set the pixel at the corresponding position in the fourth intermediate image as a background pixel. The first intermediate image is obtained by multiplying the second intermediate image with the fourth intermediate image.
6. The method for locating the center of a cross-shaped target as described in claim 1, characterized in that, The method for locating the center of the crosshair target also includes: Iterate through each initial grayscale image in the video stream sequence and perform the following operations: Add the current frame to the latest detection group; If a target rectangle exists in the current frame, the intersection-union ratio of the target rectangle in the current frame with the target rectangle in each frame of each detection group is calculated. If there is an intersection-over-union ratio (IoU) greater than zero, the current frame is added to the detection group with the largest IoU average; otherwise, the current frame is added to the newly created detection group. If the number of frames in the latest detection group is equal to the frame number threshold, then the detection group with the most frames is selected as the best detection group. The center of the crosshair target corresponding to the latest detection group is determined based on the center of the target rectangle block corresponding to at least one frame in the best detection group. A new detection group is created, and all detection groups are deleted.
7. The method for locating the center of a cross-shaped target as described in claim 6, characterized in that, The step of determining the center of the crosshair target corresponding to the latest detection group based on the center of the target rectangular block corresponding to at least one frame in the best detection group includes: The center of the target rectangle in the latest frame of the best detection group is taken as the center of the cross-shaped target in the latest detection group.
8. The method for locating the center of a cross-shaped target as described in claim 1, characterized in that, The method further includes the following steps before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If the initial grayscale image does not have a corresponding candidate region, then the initial grayscale image is downsampled, filtered, and gradient calculated to obtain the fifth intermediate image; The target region is obtained by extracting local maxima in the fifth intermediate image. The target region is expanded by image dilation. The smallest bounding rectangle of the target region is used as the candidate region corresponding to the initial grayscale image.
9. The method for locating the center of a cross-shaped target as described in claim 8, characterized in that, The step of downsampling, filtering, and gradient calculation of the initial grayscale image to obtain the fifth intermediate image if no corresponding candidate region exists in the initial grayscale image includes: If the initial grayscale image does not have a corresponding candidate region and the initial grayscale image is an inner field image, then the initial grayscale image is downsampled, filtered and gradient calculated to obtain the fifth intermediate image; The method further includes the following steps before the step of cropping the coarse localization image based on the initial grayscale image and the corresponding candidate region: If no candidate region exists and the initial grayscale image is an external image, the user is prompted to set a candidate region corresponding to the initial grayscale image.
10. A cross-shaped target center positioning device, characterized in that, The crosshair target center positioning device includes: The image cropping module is used to crop a coarse localization image based on the initial grayscale image and the corresponding candidate region; An image conversion module is used to generate a first intermediate image based on a coarse localization image, wherein the operation of generating the first intermediate image includes performing gradient calculation on the coarse localization image; The grid partitioning module is used to determine the target row matrix and target column matrix from the first intermediate image. Pixels at corresponding positions in the coarse localization image are used as marker points. The coarse localization image is divided into regions to obtain multiple candidate rectangular blocks. Each row matrix has an equal number of pixel rows, and each column matrix has an equal number of pixel columns. The target row matrix is a set of multiple row matrices with the largest weighted sum of pixel values, and the target column matrix is a set of multiple column matrices with the largest weighted sum of pixel values. The candidate rectangular blocks include a main body composed of unmarked points. The score calculation module is used to calculate the center score of each candidate rectangular block according to preset rules. The preset rules include: the smaller the difference in average pixel value between the current candidate rectangular block and its horizontal and vertical adjacent candidate rectangular blocks, and the larger the difference in average pixel value with its diagonal adjacent candidate rectangular blocks, the higher the center score of the current candidate rectangular block. The center localization module is used to select the candidate rectangular block with the highest center score that is greater than the score threshold as the target rectangular block, and to use the center of the target rectangular block as the center of the cross target corresponding to the initial grayscale image.