A space object detection and tracking method

Abstract

The application discloses a space target detection and tracking method, relates to the computer vision target detection and tracking technology, and solves the technical problems of noise interference, poor tracking window adaptability, easy errors in coordinate conversion, and slow target loss recovery of the existing method. The detection process of the space point and surface target is divided into a global detection stage and a local tracking stage. In the global detection stage, candidate targets are extracted through multi-frame statistics and angle deviation screening. In the local tracking stage, real-time tracking of the target neighborhood is realized based on a dynamic search box, and the tracking stability is improved through morphological processing, size filtering, coordinate unified conversion and other strategies. Meanwhile, a closed-loop mechanism of triggering global detection when tracking fails is designed, so that the accuracy and continuity of the point and surface target detection and tracking in the whole process are ensured. The application can be widely applied to star point detection, small target tracking and other scenes.

Description

Technical Field

[0001] This invention relates to target detection and tracking methods in computer vision, specifically to a spatial target detection and tracking method applicable to real-time detection and tracking of point or small area targets. Background Technology

[0002] Target detection and tracking refers to the continuous extraction and identification of moving targets in acquired video image sequences to obtain the target's motion parameters (such as centroid position, velocity, acceleration, etc.) for further processing and analysis, thereby achieving an understanding of the target's behavior to complete other tasks.

[0003] For example, Chinese Patent CN106651904A provides a method for capturing and tracking multiple spatial targets across a wide size range. Primarily targeting multi-target detection and tracking scenarios, it synchronously extracts point and area targets from input video images. Stars are removed using star catalog information through star map matching, or the centroid position of unknown celestial bodies in the image is identified using motion vector differences through inter-frame matching. Target tracking is then performed synchronously on the centroid positions of the identified images of multiple unknown celestial bodies. Chinese Patent CN114820801A provides a method for detecting spatial targets, primarily targeting various complex detection scenarios. It preprocesses the image using morphological methods, detecting the bounding rectangles of all star points within the field of view based on dilation and erosion processing, and deriving the features of the star point regions, eliminating noise and lunar interference. Inter-frame star point matching is completed based on the slow transformation of star point features between frames. Abnormal matches are eliminated through trajectory feature accumulation, and trajectory descriptions of all star points and spatial targets are obtained after trajectory association. The position of the spatial target is determined by assigning scores to the star points.

[0004] For point-and-surface targets (i.e., spatial targets) that are characterized by small size, high contrast, susceptibility to noise interference, and easy deviation of motion trajectory, traditional target detection and tracking methods typically suffer from the following technical problems:

[0005] 1. During the global detection phase, the movement directions of the background and the target are not distinguished, which can easily lead to background noise being misjudged as the target.

[0006] 2. The search box used in the tracking phase has a fixed size, which cannot adapt to changes in the target size. Therefore, the tracking window may be too large or too small, which may lead to the introduction of too much noise or loss of the target.

[0007] 3. The logic for transforming local and global coordinates during the tracking phase is relatively complex. When the target size is slightly larger, coordinate offset issues are prone to occur, resulting in lower tracking accuracy.

[0008] 4. If tracking fails during the tracking phase, it is difficult to resume tracking in a timely manner after the target is lost.

[0009] These issues result in insufficient detection accuracy and tracking robustness of existing detection and tracking methods. Summary of the Invention

[0010] The purpose of this invention is to solve the technical problems of existing technologies, such as noise interference, poor adaptability of tracking windows, easy errors in coordinate transformation, and difficulty in recovering lost targets, and to provide a spatial target detection and tracking method.

[0011] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0012] A spatial target detection and tracking method, characterized by the following steps:

[0013] S0, Initialization:

[0014] Set up an image acquisition system and configure detection and tracking parameters;

[0015] A model is created for the spatial target to be detected and tracked, resulting in a star point structure model corresponding to the spatial target. The star point structure model includes the centroid coordinates, circumscribed rectangle, trajectory list, and tracking markers of the star point structure. The initial state of the tracking markers is invalid.

[0016] S1, Image Acquisition:

[0017] S1.1, At least two frames of images are acquired using an image acquisition system;

[0018] S1.2, Determine whether the tracking markers in the star point structure model are valid: if the tracking markers are invalid, proceed to step S2; if the tracking markers are valid, proceed to step S3.

[0019] S2, Global Detection:

[0020] Based on at least two frames of images obtained in step S1, target star points are filtered. If no target star points are filtered, step S4 is executed directly. If target star points are filtered, the target star points are added to the trajectory list of the star point structure model, the tracking markers are updated to be valid, and then step S4 is executed.

[0021] S3, Local Tracking:

[0022] Take one frame from the at least one frame obtained in step S1. Calculate a local cropping window for this frame based on the target star and detection / tracking parameters. Within the local cropping window, filter local star points in the neighborhood of the target star to obtain local star points. Match this local star point with the target star point. If the match fails, clear the trajectory list in the star point structure model, update the tracking marker to invalid, and then return to step S2 to re-perform global detection based on this frame. If the match succeeds, update the star point structure model based on the corresponding local star point, use this local star point as the spatial target to be detected and tracked, and then proceed to step S4.

[0023] S4. Determine whether further detection and tracking are needed. If so, return to step S1; otherwise, complete the detection and tracking of the spatial target.

[0024] Furthermore, in step S0, detection and tracking parameters are configured based on the acquisition parameters of the image acquisition system and the requirements for spatial target detection and tracking;

[0025] In step S2, the target star point screening includes global threshold segmentation, connected component screening, and angle deviation verification;

[0026] In step S3, local star point screening includes local threshold segmentation and morphological processing.

[0027] Further, in step S0, the detection and tracking parameters include the minimum pixel size of the star point STAR_MIN_SIZE, the detection interval DETECT_SKIP_FRAME, the angle deviation threshold ANGLE_DEV_THRESH, the dynamic search box scaling factor SEARCH_WIN_SCALE, the minimum search box size MIN_SEARCH_WIN, the maximum search box size MAX_SEARCH_WIN, the window edge padding value STAR_WINDOW_PAD, the tracking stage threshold STAR_THRESH1, the struct element size STAR_MORPH_KERNEL_SIZE, the number of erosion iterations STAR_ERODE_ITER, the number of dilation iterations STAR_DILATE_ITER, the large target threshold STAR_TRACK_MAX_SIZE, the maximum inter-frame matching distance STAR_TRACK_DIST, and the minimum size ratio TRACK_MIN_SIZE_RATIO.

[0028] The minimum pixel size of the star point, STAR_MIN_SIZE, ranges from 2 to 5; the detection interval, DETECT_SKIP_FRAME, ranges from 1 to 10; the angle deviation threshold, ANGLE_DEV_THRESH, ranges from 20° to 40°; the dynamic search box scaling factor, SEARCH_WIN_SCALE, ranges from 5 to 12; the tracking stage threshold, STAR_THRESH1, ranges from 180 to 255; the large target threshold, STAR_TRACK_MAX_SIZE, ranges from 25 to 35; and the maximum inter-frame matching distance, STAR_TRACK_DIST, ranges from 20 to 30.

[0029] The values ​​of minimum search box size MIN_SEARCH_WIN, maximum search box size MAX_SEARCH_WIN, window edge padding value STAR_WINDOW_PAD, structuring element size STAR_MORPH_KERNEL_SIZE, erosion iteration count STAR_ERODE_ITER, dilation iteration count STAR_DILATE_ITER, and minimum size ratio TRACK_MIN_SIZE_RATIO are determined based on actual requirements and the image resolution of the image acquisition system.

[0030] Furthermore, in step S0, the image acquisition system is built on the Hi3516DV500 hardware platform, with an image resolution of 1920×1080 and a frame rate of 25fps.

[0031] The specific detection and tracking parameters are as follows: minimum pixel size STAR_MIN_SIZE=3, detection interval DETECT_SKIP_FRAME=5, angle deviation threshold ANGLE_DEV_THRESH=30.0°, dynamic search box scaling factor SEARCH_WIN_SCALE=9.0, minimum search box size MIN_SEARCH_WIN=30, maximum search box size MAX_SEARCH_WIN=1200, window edge padding value STAR_WINDOW_PAD=2, tracking stage threshold STAR_THRESH1=180, struct element size STAR_MORPH_KERNEL_SIZE=5, erosion iteration count STAR_ERODE_ITER=1, dilation iteration count STAR_DILATE_ITER=1, large target threshold STAR_TRACK_MAX_SIZE=32, maximum inter-frame matching distance STAR_TRACK_DIST=25, and minimum size ratio TRACK_MIN_SIZE_RATIO=0.25.

[0032] Furthermore, in step S1, the image acquisition system acquires at least one frame of real-time image in YUV420SP format;

[0033] A real-time image in YUV420SP format is converted into a single-channel matrix containing only the Y component and the luminance channel by using a memory copy method. As an image, the conversion formula is as follows:

[0034] ;

[0035] in, Let be the pixel value in the i-th row and j-th column of the Y-component single-channel matrix. i and j are two positional variables: i is the image row number (0 ≤ i ≤ H-1), and j is the image column number (0 ≤ j ≤ W-1). W is the width of the real-time image, and H is the image height. For the Y component virtual address in YUV420SP format; The coordinate system of the image is based on the origin at the top left corner, with the x-axis pointing to the right and the y-axis pointing down.

[0036] Furthermore, step S2 specifically includes:

[0037] S2.1, Global Threshold Segmentation

[0038] Take one frame from the image obtained in step S1 and calculate the Y component single-channel matrix. global average brightness The calculation formula is as follows:

[0039] ;

[0040] Where W and H are the width and height of the image, respectively. The pixel value in the i-th row and j-th column of the Y-component single-channel matrix;

[0041] Calculate the adaptive threshold ;

[0042] Adaptive threshold Limited to Within the interval, the detection threshold is obtained. :

[0043] ;

[0044] Based on detection threshold For the single-channel matrix of the Y component Binarization is performed to obtain the binarized matrix for the detection stage. ;

[0045] ;

[0046] in, , where is the pixel value in the i-th row and j-th column of the binarization matrix during the detection phase, 255 represents candidate star points, and 0 represents background star points;

[0047] S2.2, Connected Component Filtering

[0048] For binary matrix Perform 8-connected component processing to obtain star point information; the star point information includes the location and area of ​​the connected component. and the centroid coordinates of the connected domain ;

[0049] centroid coordinates The calculation formula is as follows:

[0050] ;

[0051] ;

[0052] in, Let x and y be the set of pixels in the connected component, where x and y are the coordinates of the pixels.

[0053] Based on the minimum pixel size STAR_MIN_SIZE of the star points, all star point information was analyzed, and the following were selected. Connected components are used to obtain a list of candidate star points;

[0054] S2.3, Statistical analysis of multi-frame trajectories

[0055] Each collection Frame, for each frame, the centroid coordinates of the candidate star points in the candidate star point list. Statistical analysis was performed to obtain the trajectories of the candidate stars, and the motion angles of the candidate stars were calculated. The calculation formula is as follows:

[0056] ;

[0057] Rotating counterclockwise with the angle as the positive direction, if the calculated angle of motion... Then let This places it within the 0~360° range;

[0058] S2.4, Calculate the ring angle deviation

[0059] Calculate the background motion angle of the background stars The calculation formula is as follows:

[0060] ;

[0061] Where N is the number of background stars. Let i be the motion angle of the i-th background star point;

[0062] If the calculated background motion angle Then let This places it within the 0~360° range;

[0063] Calculate the annular angular deviation between the motion angle of each candidate star point and the background motion angle. The calculation formula is as follows:

[0064] ; Make a judgment: If the number of candidate stars or background stars N=0, terminate the current round of global detection and proceed to step S4; If the variance of the motion angle of the background stars is greater than the preset discrete threshold, then all candidate stars that meet the size requirements are temporarily designated as target stars. If the background motion angle If there is a calculation failure or data anomaly, retain all valid candidate star points and proceed to step S3; if there are no valid candidate star points, proceed to step S4.

[0065] S2.5, Angle Deviation Verification

[0066] Based on the angular deviation threshold ANGLE_DEV_THRESH, the annular angular deviation of each candidate star point is... Analysis:

[0067] like If the candidate star point is determined to be background noise, it is removed from the candidate star point list, and the next candidate star point is analyzed until all candidate star points in the candidate star point list have been analyzed. If the target star point is still not obtained, step S4 is executed.

[0068] like If the candidate star point is selected as the target star point, it is added to the trajectory list of the star point structure model, and the centroid coordinates of the star point structure model are updated accordingly. Circumscribed rectangle Execute step S3; where, and The width and height of the bounding rectangle.

[0069] Furthermore, step S3 specifically includes:

[0070] S3.1, Dynamic Search Box Calculation

[0071] Take one frame from the image obtained in step S1, and calculate the dynamic search box size of the frame based on the bounding rectangle of the target star point, the minimum search box size MIN_SEARCH_WIN, the maximum search box size MAX_SEARCH_WIN, and the dynamic search box scaling factor SEARCH_WIN_SCALE. The calculation formula is as follows:

[0072] ;

[0073] S3.2, Partial Clipping Window

[0074] Centered on the centroid coordinates of the target star, based on the dynamic search box size Clipping a partial window using the STAR_WINDOW_PAD window edge padding value. Local window coordinates The calculation method is as follows:

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] Where W and H are the width and height of the image, This is a round-down operation;

[0080] S3.3, Local Threshold Segmentation and Morphological Processing

[0081] For local windows Perform threshold segmentation based on the tracking phase threshold STAR_THRESH1. Binarization is performed to obtain the binarized matrix for the tracking stage. ;

[0082] ;

[0083] in, The pixel value in the i-th row and j-th column of the binarized matrix during the tracking phase;

[0084] Based on the structuring element size STAR_MORPH_KERNEL_SIZE, the number of erosion iterations STAR_ERODE_ITER, and the number of dilation iterations STAR_DILATE_ITER, the binarized matrix in the tracking stage is... Perform morphological processing; the methods for morphological processing are as follows:

[0085] Binarized matrix after erosion treatment ;

[0086] Binarized matrix after dilation ;

[0087] Where K is a rectangular structural element with dimensions STAR_MORPH_KERNEL_SIZE × STAR_MORPH_KERNEL_SIZE. ⊕ is the erosion operator, and ⊕ is the dilation operator;

[0088] S3.4, Local Starpoint Matching

[0089] Binarized matrix after dilation Perform 8-connected component processing to obtain local star point information; the local star point information includes the location and area of ​​the local connected component. and the centroid coordinates of the locally connected domain ;

[0090] Determine the size of the target star point based on the large target threshold STAR_TRACK_MAX_SIZE:

[0091] like If the local connected region with the largest area within the local window is selected, then the target star point is selected as the matching star point.

[0092] like Then, the locally connected components within the local window are extracted as local star points, and the centroid coordinates of the local star points are converted to global centroid coordinates, as follows:

[0093] ;

[0094] ;

[0095] ;

[0096] in, and The global centroid coordinates of the local star point. and These are the bounding rectangles of the local star point and the global bounding rectangle, respectively;

[0097] The Euclidean distance d between the local star point and the target star point is calculated as follows:

[0098] ;

[0099] Analysis of local star erosion points based on the maximum inter-frame matching distance STAR_TRACK_DIST and the minimum size ratio TRACK_MIN_SIZE_RATIO:

[0100] like If the local star point is determined to be the target star point, the match is considered successful.

[0101] Otherwise, analyze the next local star point until the local star point analysis is completed; If there are more than two local star points in the neighborhood, and multiple local star points still appear in the next frame, then global detection is triggered, the trajectory list in the star point structure model is cleared, the tracking marker is updated to invalid, and then the process returns to step S2 to re-select target star points and perform global detection.

[0102] S3.5, Determine whether the image in this frame is successfully matched:

[0103] If a match fails, a global detection is triggered, the trajectory list in the star point structure model is cleared, the tracking markers are updated to invalid, and then the process returns to step S2 to re-filter target star points; if a match succeeds, the centroid coordinates of the star point structure model are updated based on the corresponding local star point. Circumscribed rectangle The tracking list is updated to be valid, the local star point is selected as the spatial target to be tracked, and then step S4 is executed.

[0104] Further, in step S1.1, the image acquisition system acquires 5 frames of real-time images in YUV420SP format;

[0105] In steps S2.1 and S3.1, the last frame of the five real-time images obtained in step S1 is taken.

[0106] Furthermore, in step S2.1, the interval .

[0107] Compared with the prior art, the present invention has the following beneficial effects:

[0108] 1. This invention provides a spatial target detection and tracking method, which designs a phased detection and tracking approach: in the global detection phase, multi-frame statistics and angle deviation filtering are used to effectively distinguish the target from background noise; in the local tracking phase, the target neighborhood is focused to reduce the amount of computation and improve real-time performance; an adaptive threshold segmentation method is proposed, which can effectively separate the target star points; this invention has the characteristics of noise resistance, high tracking accuracy, and strong robustness, and can be widely used in scenarios such as star point detection and small target tracking, and can achieve high-precision and high-robust detection and tracking of point and surface targets.

[0109] 2. The spatial target detection and tracking method provided by the present invention designs a dynamic search box adaptation, dynamically adjusts the tracking window based on the target size, solves the problem of poor adaptability of fixed windows, covers the target movement range, and can adapt to changes in target size.

[0110] 3. The spatial target detection and tracking method provided by this invention designs a unified coordinate transformation, optimizes the existing local-global coordinate transformation logic, avoids large-size target coordinate offset, and improves tracking accuracy.

[0111] 4. The space target detection and tracking method provided by this invention has a closed-loop re-examination mechanism. After tracking failure, a global re-examination is automatically triggered, and the target can be quickly recovered after being lost, thus improving the continuity of tracking.

[0112] 5. The present invention provides a spatial target detection and tracking method. The detection and tracking parameters can be configured according to actual needs. The core parameters used for detection and tracking (threshold, size, distance, angle, etc.) support flexible debugging and can adapt to the point and surface target detection and tracking needs in different scenarios. Attached Figure Description

[0113] Figure 1 This is a flowchart of an embodiment of a space target detection and tracking method according to the present invention. Detailed Implementation

[0114] To make the objectives, advantages, and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0115] This invention addresses the detection and tracking of point and surface spatial targets, such as stars. It divides the detection process into a global detection stage and a local tracking stage. In the global detection stage, candidate targets are extracted through multi-frame statistics and angle deviation screening. In the local tracking stage, real-time tracking of the target neighborhood is achieved based on a dynamic search box. This invention is adapted to the characteristics of point and surface targets and is suitable for real-time detection and tracking of point and small surface targets in scenarios such as security monitoring, astronomical observation, and industrial visual inspection.

[0116] like Figure 1 As shown, this embodiment of the invention provides a space target detection and tracking method, taking star point detection and tracking as an application scenario. The detection and tracking of space targets is divided into a global detection stage and a local tracking stage, with the initial stage being the global detection stage. The specific steps include the following:

[0117] S0, Initialization:

[0118] An image acquisition system was built, and detection and tracking parameters were configured based on the acquisition parameters of the image acquisition system and the requirements for spatial target detection and tracking. The spatial target to be detected and tracked was modeled to obtain a star-point structure model corresponding to the spatial target. Specifically:

[0119] An image acquisition system was built based on the hi3516dv500 hardware platform, with an image resolution of 1920×1080 and a frame rate of 25fps.

[0120] The detection and tracking parameters include: minimum pixel size STAR_MIN_SIZE = 3 (adjustable from 2 to 5, used to filter small area noise), detection interval DETECT_SKIP_FRAME = 5 (adjustable from 1 to 10), angle deviation threshold ANGLE_DEV_THRESH = 30.0° (adjustable from 20° to 40°, used to distinguish the target from the background movement direction), dynamic search box scaling factor SEARCH_WIN_SCALE = 9.0 (adjustable from 5 to 12), minimum search box size MIN_SEARCH_WIN = 30, maximum search box size MAX_SEARCH_WIN = 1200, and window edge padding value STAR_WIN. DOW_PAD=2, tracking stage threshold STAR_THRESH1=180 (adjustable from 180 to 255), structural element size STAR_MORPH_KERNEL_SIZE=5, erosion iteration count STAR_ERODE_ITER=1, dilation iteration count STAR_DILATE_ITER=1, large target threshold STAR_TRACK_MAX_SIZE=32 (adjustable from 25 to 35), maximum inter-frame matching distance STAR_TRACK_DIST=25 (adjustable from 20 to 30, used to adapt to the target movement range), minimum size ratio TRACK_MIN_SIZE_RATIO=0.25;

[0121] The values ​​of the minimum search box size MIN_SEARCH_WIN, maximum search box size MAX_SEARCH_WIN, window edge padding value STAR_WINDOW_PAD, struct element size STAR_MORPH_KERNEL_SIZE, erosion iteration count STAR_ERODE_ITER, dilation iteration count STAR_DILATE_ITER, and minimum size ratio TRACK_MIN_SIZE_RATIO can be determined according to the actual task requirements and the image resolution of the image acquisition system.

[0122] A model is created for the spatial target to be detected and tracked, resulting in a star point structure model corresponding to the spatial target. The star point structure model includes the centroid coordinates, circumscribed rectangle, trajectory list, and tracking markers of the star point structure. Detection and tracking are performed based on this star point structure model. The initial state of the tracking markers is invalid.

[0123] S1, Image Acquisition:

[0124] S1.1, real-time images in YUV420SP format are acquired through the image acquisition system, containing 5 consecutive frames of images;

[0125] A real-time image in YUV420SP format is converted into a single-channel matrix containing only the Y component and the luminance channel by using a memory copy method. The conversion formula for the image used later is as follows:

[0126] ;

[0127] in, Let be the pixel value in the i-th row and j-th column of the Y-component single-channel matrix. i and j are two positional variables: i is the image row number (0 ≤ i ≤ H-1), and j is the image column number (0 ≤ j ≤ W-1). W is the image width, and H is the image height. This is the virtual address of the Y component in YUV420SP format. The coordinate system of the image is based on the origin at the top left corner, with the x-axis pointing to the right and the y-axis pointing down.

[0128] Extracting the luminance channel can reduce the overall computational load, while direct memory copying can achieve efficient conversion.

[0129] The YUV420SP data storage arrangement is as follows: the image luminance Y component is stored sequentially in row-major order, occupying the first W×H bytes of memory space in the image, with pixel arrangement corresponding one-to-one with image spatial position; the chrominance UV components are stored after the Y component data, using a 2×2 pixel unit sampling and UV alternating arrangement mode, with every 4 adjacent pixels sharing a set of UV chrominance data, and the overall resolution of the chrominance data is 1 / 4 of that of the luminance data. This invention extracts only the Y component with unique position mapping and regular data for image processing, discarding redundant chrominance components, reducing the computational load and improving the real-time performance of the algorithm without sacrificing star detection accuracy.

[0130] S1.2, Determine whether the tracking markers in the star point structure model are valid: If the tracking markers are invalid and the system is in the global detection stage, then proceed to step S2; if the tracking markers are valid and the system is in the local tracking stage, then proceed to step S3.

[0131] S2, Global Detection Phase:

[0132] Based on the obtained 5 frames of images (i.e., processed every 4 frames), global detection is performed, sequentially through global threshold segmentation, connected component filtering, and angle deviation verification to filter target star points. If no target star point is found, step S4 is executed directly; if a target star point is found, it is added to the trajectory list of the star point structure model, the tracking marker is updated to valid, and then step S4 is executed. Specifically:

[0133] S2.1, Global Threshold Segmentation

[0134] Take the last frame from the 5 frames obtained in step S1.1, and process the Y component single-channel matrix. Perform adaptive global thresholding segmentation and calculate the single-channel matrix of the Y component. global average brightness The calculation formula is as follows:

[0135] ;

[0136] Where W and H are the width and height of the image, respectively. The pixel value in the i-th row and j-th column of the Y-component single-channel matrix;

[0137] Calculate the adaptive threshold ;

[0138] Adaptive threshold Limited to Within the interval, the detection threshold is obtained. :

[0139] ;

[0140] Based on detection threshold For the single-channel matrix of the Y component Binarization is performed to obtain the binarized matrix for the detection stage. ;

[0141] ;

[0142] in, , where is the pixel value in the i-th row and j-th column of the binarization matrix during the detection phase, 255 represents candidate star points (foreground), and 0 represents background star points;

[0143] S2.2, Connected Component Filtering

[0144] For binary matrix Perform 8-connected component processing to obtain star point information; the star point information includes the location and area of ​​the connected component. and the centroid coordinates of the connected domain ;

[0145] centroid coordinates The calculation formula is as follows:

[0146] ;

[0147] ;

[0148] in, Let x and y be the set of pixels in the connected component, where x and y are the coordinates of the pixels.

[0149] Based on the minimum pixel size STAR_MIN_SIZE of the star points, all star point information was analyzed, and the following were selected. Connected components are used to obtain a list of candidate star points;

[0150] S2.3, Statistical analysis of multi-frame trajectories

[0151] Each collection Frame, for each frame, the centroid coordinates of the candidate star points in the candidate star point list. Statistical analysis was performed to obtain the trajectories of the candidate stars, and the motion angles of the candidate stars were calculated. The calculation formula is as follows:

[0152] ;

[0153] Rotating counterclockwise with the angle as the positive direction, if the calculated angle of motion... Then let This places it within the 0~360° range;

[0154] S2.4, Calculate the ring angle deviation

[0155] Calculate the background motion angle of the background stars The calculation formula is as follows:

[0156] ;

[0157] Where N is the number of background stars. Let i be the motion angle of the i-th background star point;

[0158] If the calculated background motion angle Then let This places it within the 0~360° range;

[0159] Calculate the annular angular deviation between the motion angle of each candidate star point and the background motion angle. The calculation formula is as follows:

[0160] ; Make a judgment: If the number of candidate star points or background star points N=0, it is considered that there is no valid motion data, and it is determined that there are no valid target star points in the current frame. The global detection in this round is terminated, and step S4 is executed. If the variance of the motion angle of the background stars is greater than the preset discrete threshold, it is determined that the background direction distribution is discrete and there is no uniform background motion law. Then, all candidate stars that meet the size requirements are temporarily designated as target stars. If the background motion angle If there is a calculation failure or data anomaly, retain all valid candidate star points and proceed to step S3. If there are no valid candidate star points, proceed to step S4 to avoid meaningless filtering that may lead to missed detection of the target.

[0161] S2.5, Angle Deviation Verification

[0162] Based on the angular deviation threshold ANGLE_DEV_THRESH, the annular angular deviation of each candidate star point is... Analysis:

[0163] like If the candidate star point is determined to be background noise, it is removed from the candidate star point list, and the next candidate star point is analyzed until all candidate star points in the candidate star point list have been analyzed. If the target star point is still not obtained, step S4 is executed.

[0164] like If the candidate star point is selected as the target star point, it is added to the trajectory list of the star point structure model, and the centroid coordinates of the star point structure model are updated accordingly. Circumscribed rectangle Execute step S3. Wherein, and The width and height of the bounding rectangle.

[0165] For example, the centroid coordinates of the target star point are (cx, cy) = (960, 540), and the size of the circumscribed rectangle is 32×32.

[0166] S3, Local Tracking:

[0167] The last frame is taken from the five obtained images. A local cropping window is calculated based on the target star detection and tracking parameters. Local tracking is performed on the neighborhood of the target star within this window. Local star points are filtered through local thresholding and morphological processing to obtain local star points. These local star points are then matched with the target star point. Further judgment is made: if the match fails, the trajectory list in the star point structure model is cleared, the tracking marker is updated to invalid, and the process returns to step S2. Global detection and target star point filtering are then performed again based on this frame. If the match succeeds, the star point structure model is updated based on the corresponding local star point, and this local star point is used as the spatial target to be detected and tracked (the new target star point). Then, step S4 is executed. Specifically:

[0168] S3.1, Dynamic Search Box Calculation

[0169] Process the last frame of image from step S1.1. Based on the bounding rectangle of the target star point, the minimum search box size MIN_SEARCH_WIN, the maximum search box size MAX_SEARCH_WIN, and the dynamic search box scaling factor SEARCH_WIN_SCALE, calculate the dynamic search box size. The calculation formula is as follows:

[0170]

[0171] ;

[0172] S3.2, Partial Clipping Window

[0173] Centered on the centroid coordinates of the target star, based on the dynamic search box size Clipping a partial window using the STAR_WINDOW_PAD window edge padding value. Local window coordinates The calculation method is as follows:

[0174]

[0175] ;

[0176]

[0177] ;

[0178]

[0179] ;

[0180]

[0181] ;

[0182] Where W and H are the width and height of the image, This is a rounding down operation; the local window size is 292×292 at this point.

[0183] S3.3, Local Threshold Segmentation and Morphological Processing

[0184] For local windows Perform threshold segmentation based on the tracking phase threshold STAR_THRESH1. Binarization is performed to obtain the binarized matrix for the tracking stage. ;

[0185] ;

[0186] in, The pixel value in the i-th row and j-th column of the binarized matrix during the tracking phase;

[0187] Based on the structuring element size STAR_MORPH_KERNEL_SIZE, the number of erosion iterations STAR_ERODE_ITER, and the number of dilation iterations STAR_DILATE_ITER, the binarized matrix in the tracking stage is... Morphological processing was performed (5×5 rectangular structural elements, 1 erosion + 1 dilation); the morphological processing method is as follows:

[0188] Binarized matrix after erosion treatment ;

[0189] Binarized matrix after dilation ;

[0190] Where K is a rectangular structural element with dimensions STAR_MORPH_KERNEL_SIZE × STAR_MORPH_KERNEL_SIZE. For the corrosion operator, For dilation operators;

[0191] S3.4, Local Starpoint Matching

[0192] Binarized matrix after dilation Perform 8-connected component processing to obtain local star point information; the local star point information includes the location and area of ​​the local connected component. and the centroid coordinates of the locally connected domain ;

[0193] Determine the size of the target star point based on the large target threshold STAR_TRACK_MAX_SIZE:

[0194] like If the local connected region with the largest area within the local window is selected, then the target star point is selected as the matching star point.

[0195] like Then, the locally connected components within the local window are extracted as local star points, and the centroid coordinates of the local star points are converted to global centroid coordinates, as shown in the star point target above:

[0196] ;

[0197] ;

[0198] ;

[0199] in, and The global centroid coordinates of the local star point. and These are the bounding rectangles of the local star point and the global bounding rectangle, respectively;

[0200] Calculate the Euclidean distance d between the local star point and the target star point:

[0201]

[0202] ;

[0203] Analysis of local star points is based on the maximum inter-frame matching distance STAR_TRACK_DIST and the minimum size ratio TRACK_MIN_SIZE_RATIO:

[0204] like If the local star point is determined to be the target star point, the match is considered successful.

[0205] Otherwise, analyze the next local star point until the local star point analysis is completed; If there are more than two local star points in the neighborhood, and multiple local star points still appear in the next frame, it is considered that the continuous tracking of the target star point cannot be completed. Then, global detection is triggered, the trajectory list in the star point structure model is cleared, the tracking mark is updated to invalid, and then the process returns to step S2 to re-select the target star points and perform global detection.

[0206] S3.5, Determine whether the image in this frame is successfully matched:

[0207] If a match fails, a global detection is triggered, the trajectory list in the star point structure model is cleared, the tracking marker is updated to invalid, and then the process returns to step S2 to re-filter the target star points.

[0208] If a match is successful, the centroid coordinates of the star point structure model are updated based on the corresponding local star point. Circumscribed rectangle The tracking list is updated to be valid, the local star point is selected as the spatial target to be tracked, and then step S4 is executed.

[0209] S4. Determine whether further detection and tracking are needed. If so, return to step S1; otherwise, complete the detection and tracking of the spatial target.

[0210] In this embodiment, the tracking accuracy of star targets can reach ±1 pixel, the frame rate is maintained at 25fps, and global detection can be triggered within 1 frame after tracking failure to resume target tracking. Therefore, the method of the present invention can effectively solve the problems of noise interference and tracking offset of star targets.

[0211] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A method for detecting and tracking space targets, characterized in that, The steps include the following: S0, Initialization: An image acquisition system is set up, and detection and tracking parameters are configured. The detection and tracking parameters include the minimum pixel size of star points (STAR_MIN_SIZE), the detection interval (DETECT_SKIP_FRAME), and the angle deviation threshold (ANGLE_DEV_THRESH). A model is created for the spatial target to be detected and tracked, resulting in a star point structure model corresponding to the spatial target. The star point structure model includes the centroid coordinates, circumscribed rectangle, trajectory list, and tracking markers of the star point structure. The initial state of the tracking markers is invalid. S1, Image Acquisition: S1.1, At least two frames of images are acquired using an image acquisition system; the images are real-time images in YUV420SP format; A real-time image in YUV420SP format is converted into a single-channel matrix containing only the Y component, using a memory copy method. As an image, the conversion formula is as follows: ； in, Let be the pixel value in the i-th row and j-th column of the Y-component single-channel matrix. i and j are two positional variables: i is the image row number (0 ≤ i ≤ H-1), and j is the image column number (0 ≤ j ≤ W-1). W is the image width, and H is the image height. For the Y component virtual address in YUV420SP format; The coordinate system of the image is based on the origin at the top left corner, with the x-axis pointing to the right and the y-axis pointing down; S1.2, Determine whether the tracking markers in the star point structure model are valid: if the tracking markers are invalid, proceed to step S2; if the tracking markers are valid, proceed to step S3. S2, Global Detection: S2.1, Global Threshold Segmentation; Take one frame from the image obtained in step S1 and calculate the single-channel matrix of the Y component. global average brightness The calculation formula is as follows: ； Where W and H are the width and height of the image, respectively. The pixel value in the i-th row and j-th column of the Y-component single-channel matrix; Calculate the adaptive threshold ; Adaptive threshold Limited to Within the interval, the detection threshold is obtained. : ； Based on detection threshold For the single-channel matrix of the Y component Binarization is performed to obtain the binarized matrix for the detection stage. ; ； in, , where is the pixel value in the i-th row and j-th column of the binarization matrix during the detection phase, 255 represents candidate star points, and 0 represents background star points; S2.2, Connected component filtering; For binary matrix Perform 8-connected component processing to obtain star point information; the star point information includes the position and area of ​​the connected component. and the centroid coordinates of the connected domain ; The centroid coordinates The calculation formula is as follows: ； ； in, Let x and y be the set of pixels in the connected component, where x and y are the coordinates of the pixels. Based on the minimum pixel size STAR_MIN_SIZE of the star points, all star point information was analyzed, and the following were selected. Connected components are used to obtain a list of candidate star points; S2.3, statistical analysis of multi-frame trajectories; Each collection Frame, for each frame, the centroid coordinates of the candidate star points in the candidate star point list. Statistical analysis was performed to obtain the trajectories of the candidate stars, and the motion angles of the candidate stars were calculated. The calculation formula is as follows: ； Rotating counterclockwise with the angle as the positive direction, if the calculated angle of motion... Then let This places it within the 0~360° range; S2.4, Calculate the ring angle deviation; Calculate the background motion angle of the background stars The calculation formula is as follows: ； Where N is the number of background stars. Let i be the motion angle of the i-th background star point; If the calculated background motion angle Then let This places it within the 0~360° range; Calculate the annular angular deviation between the motion angle of each candidate star point and the background motion angle. The calculation formula is as follows: ； Make a judgment: If the number of candidate stars or background stars N=0, terminate the current round of global detection and proceed to step S4; If the variance of the motion angle of the background stars is greater than the preset discrete threshold, then all candidate stars that meet the size requirements are temporarily designated as target stars. If the background motion angle If there is a calculation failure or data anomaly, retain all valid candidate star points and proceed to step S3; if there are no valid candidate star points, proceed to step S4. S2.5, Angle Deviation Verification; Based on the angular deviation threshold ANGLE_DEV_THRESH, the annular angular deviation of each candidate star point is... Analysis: like If the candidate star point is determined to be background noise, it is removed from the candidate star point list, and the next candidate star point is analyzed until all candidate star points in the candidate star point list have been analyzed. If the target star point is still not obtained, step S4 is executed. like If the candidate star point is selected as the target star point, it is added to the trajectory list of the star point structure model, and the centroid coordinates of the star point structure model are updated accordingly. Circumscribed rectangle Execute step S3; where, and Let the width and height of the bounding rectangle be the values ​​of the bounding rectangle. S3, Local Tracking: Take one frame from the at least one frame obtained in step S1. Calculate a local cropping window for this frame based on the target star and the detection and tracking parameters. Within the local cropping window, filter local star points in the neighborhood of the target star to obtain local star points. Match this local star point with the target star point. If the match fails, clear the trajectory list in the star point structure model, update the tracking marker to invalid, and then return to step S2 to re-perform global detection based on this frame. If the match succeeds, update the star point structure model based on the corresponding local star point, use this local star point as the spatial target to be detected and tracked, and then proceed to step S4. S4. Determine whether further detection and tracking are needed. If so, return to step S1; otherwise, complete the detection and tracking of the spatial target.

2. The space target detection and tracking method according to claim 1, characterized in that: In step S0, the detection and tracking parameters are configured based on the acquisition parameters of the image acquisition system and the requirements for spatial target detection and tracking; In step S3, the local star point screening includes local threshold segmentation and morphological processing.

3. The space target detection and tracking method according to claim 2, characterized in that: In step S0, the detection and tracking parameters also include the dynamic search box scaling factor SEARCH_WIN_SCALE, the minimum search box size MIN_SEARCH_WIN, the maximum search box size MAX_SEARCH_WIN, the window edge padding value STAR_WINDOW_PAD, the tracking stage threshold STAR_THRESH1, the structuring element size STAR_MORPH_KERNEL_SIZE, the number of erosion iterations STAR_ERODE_ITER, the number of dilation iterations STAR_DILATE_ITER, the large target threshold STAR_TRACK_MAX_SIZE, the maximum inter-frame matching distance STAR_TRACK_DIST, and the minimum size ratio TRACK_MIN_SIZE_RATIO. The minimum pixel size STAR_MIN_SIZE ranges from 2 to 5; the detection interval DETECT_SKIP_FRAME ranges from 1 to 10; the angle deviation threshold ANGLE_DEV_THRESH ranges from 20° to 40°; the dynamic search box scaling factor SEARCH_WIN_SCALE ranges from 5 to 12; the tracking phase threshold STAR_THRESH1 ranges from 180 to 255; the large target threshold STAR_TRACK_MAX_SIZE ranges from 25 to 35; and the maximum inter-frame matching distance STAR_TRACK_DIST ranges from 20 to 30. The values ​​of the minimum search box size MIN_SEARCH_WIN, the maximum search box size MAX_SEARCH_WIN, the window edge padding value STAR_WINDOW_PAD, the structuring element size STAR_MORPH_KERNEL_SIZE, the number of erosion iterations STAR_ERODE_ITER, the number of dilation iterations STAR_DILATE_ITER, and the minimum size ratio TRACK_MIN_SIZE_RATIO are determined according to actual needs and the image resolution of the image acquisition system.

4. The space target detection and tracking method according to claim 3, characterized in that: In step S0, the image acquisition system is built on the Hi3516DV500 hardware platform, with an image resolution of 1920×1080 and a frame rate of 25fps. The specific detection and tracking parameters are as follows: minimum pixel size STAR_MIN_SIZE=3, detection interval DETECT_SKIP_FRAME=5, angle deviation threshold ANGLE_DEV_THRESH=30.0°, dynamic search box scaling factor SEARCH_WIN_SCALE=9.0, minimum search box size MIN_SEARCH_WIN=30, maximum search box size MAX_SEARCH_WIN=1200, window edge padding value STAR_WINDOW_PAD=2, tracking stage threshold STAR_THRESH1=180, struct element size STAR_MORPH_KERNEL_SIZE=5, erosion iteration count STAR_ERODE_ITER=1, dilation iteration count STAR_DILATE_ITER=1, large target threshold STAR_TRACK_MAX_SIZE=32, maximum inter-frame matching distance STAR_TRACK_DIST=25, and minimum size ratio TRACK_MIN_SIZE_RATIO=0.

25.

5. The space target detection and tracking method according to claim 4, characterized in that, Step S3 is as follows: S3.1, Dynamic search box calculation; Take one frame from the image obtained in step S1, and calculate the dynamic search box size of the frame based on the bounding rectangle of the target star point, the minimum search box size MIN_SEARCH_WIN, the maximum search box size MAX_SEARCH_WIN, and the dynamic search box scaling factor SEARCH_WIN_SCALE. The calculation formula is as follows: ； S3.2, partial clipping window; Centered on the centroid coordinates of the target star, based on the dynamic search box size Clipping a partial window using the STAR_WINDOW_PAD window edge padding value. Local window coordinates The calculation method is as follows: ； ； ； ； Where W and H are the width and height of the image, This is a round-down operation; S3.3, Local threshold segmentation and morphological processing; For local windows Perform threshold segmentation based on the tracking phase threshold STAR_THRESH1. Binarization is performed to obtain the binarized matrix for the tracking stage. ; ； in, The pixel value in the i-th row and j-th column of the binarized matrix during the tracking phase; Based on the structuring element size STAR_MORPH_KERNEL_SIZE, the number of erosion iterations STAR_ERODE_ITER, and the number of dilation iterations STAR_DILATE_ITER, the binarized matrix in the tracking stage is... Perform morphological processing; the methods for morphological processing are as follows: Binarized matrix after erosion ; Binarized matrix after dilation ; Where K is a rectangular structural element with dimensions STAR_MORPH_KERNEL_SIZE × STAR_MORPH_KERNEL_SIZE, ⊖ is the erosion operator, and ⊕ is the dilation operator; S3.4, Local Star Point Matching; Binarized matrix after dilation Perform 8-connected component processing to obtain local star point information; the local star point information includes the position and area of ​​the local connected component. and the centroid coordinates of the locally connected domain ; Determine the size of the target star point based on the large target threshold STAR_TRACK_MAX_SIZE: like If the local connected region with the largest area within the local window is selected, then the target star point is selected as the matching star point. like Then, the locally connected components within the local window are extracted as local star points, and the centroid coordinates of the local star points are converted to global centroid coordinates, as follows: ； ； ； in, and The global centroid coordinates of the local star point. and These are the bounding rectangles of the local star point and the global bounding rectangle, respectively; The Euclidean distance d between the local star point and the target star point is calculated as follows: ； Analysis of local star points is based on the maximum inter-frame matching distance STAR_TRACK_DIST and the minimum size ratio TRACK_MIN_SIZE_RATIO: like If the local star point is determined to be the target star point, the match is considered successful. Otherwise, analyze the next local star point until the local star point analysis is completed; If there are more than two local star points in the neighborhood, and multiple local star points still appear in the next frame, then global detection is triggered, the trajectory list in the star point structure model is cleared, the tracking marker is updated to invalid, and then the process returns to step S2 to re-select target star points and perform global detection. S3.5, Determine whether the image in this frame is successfully matched: If a match fails, a global detection is triggered, the trajectory list in the star point structure model is cleared, the tracking markers are updated to invalid, and then the process returns to step S2 to re-filter target star points; if a match succeeds, the centroid coordinates of the star point structure model are updated based on the corresponding local star point. Circumscribed rectangle The tracking list is updated to be valid, the local star point is selected as the spatial target to be detected and tracked, and then step S4 is executed.

6. The space target detection and tracking method according to claim 5, characterized in that: In step S1.1, the image acquisition system acquires 5 frames of real-time images in YUV420SP format; In steps S2.1 and S3.1, the last frame of the five real-time images obtained in step S1 is taken.

7. The space target detection and tracking method according to claim 5, characterized in that: In step S2.1, the interval .

Images