Monocular camera under weak texture multi-target detection and tracking algorithm and application system

Abstract

This invention provides a detection and tracking algorithm and application system for weakly textured multi-target targets using a monocular camera, belonging to the field of computer vision and video intelligent analysis technology. The method includes a detection stage and a tracking stage executed sequentially. In the detection stage, by analyzing candidate connected regions in the binarized image, a Gaussian mixture model is constructed based on the foreground pixel distribution for adhered or dense targets, and adaptive decomposition of single connected regions is performed. This invention introduces a adhered target separation method based on the Gaussian mixture model in the detection stage, performing pixel-level modeling of adhered or densely clustered target regions. Through multi-component Gaussian fitting of the foreground pixel coordinates, adaptive decomposition of single connected regions is achieved, effectively distinguishing multiple similar targets whose detection boxes merge due to spatial proximity or local occlusion, thereby reducing the false negative and false positive rates in dense scenes and providing more accurate and reliable initial detection results for subsequent multi-target tracking.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent video analysis technology, and in particular to a detection and tracking algorithm and application system for weakly textured multi-target targets using a monocular camera. Background Technology

[0002] Multi-object tracking technology is an important research direction in the field of computer vision and has been widely applied in scenarios such as video surveillance, industrial inspection, automated production, and behavior analysis. Existing multi-object tracking methods typically adopt a "detection-based tracking" paradigm, which involves first detecting objects in a single frame of an image and then correlating the detection results over time to form the object trajectory.

[0003] In related research, some methods rely on deep learning models to extract target appearance features and combine them with temporal modeling to achieve target association. For example, the invention with application number CN202511735544.7 proposes a multi-target tracking method based on the fusion of YOLO and MixFormer models. It obtains candidate targets through a target detection network and uses a Transformer structure to extract global and local appearance features of the targets, achieving cross-frame target association and position prediction. This type of method can achieve good tracking results in scenes where target appearance features are obvious, but its overall performance is highly dependent on the coverage of the training data and the discriminative power of the appearance features. In scenes where targets have highly similar appearances and weak or missing texture features, the association method based on appearance features is difficult to reliably distinguish between different targets, and is prone to identity switching or association errors.

[0004] To address the multi-target tracking problem in occluded scenes, the invention with application number CN202511579205.4 proposes a trajectory prediction-based multi-target tracking method in occluded scenes. This method models and predicts the historical trajectories of targets before occlusion, and performs identity matching based on the distance relationship between the predicted and actual detection positions after the target reappears. This method alleviates the identity loss problem caused by long-term occlusion to some extent, but its core matching criterion mainly relies on the Euclidean distance between target positions. When the number of targets is large, their spatial distribution is dense, and their distances are close, the distance differences between different matching schemes are small, which can easily lead to unstable matching results. In high-density, weak-texture scenes, there is still a risk of mismatched identities.

[0005] Furthermore, the invention with application number CN202511525698.3 discloses a multi-target tracking method that attempts to suppress the impact of target bounding box deformation on the association results by introducing attitude-aware intersection-union ratio and cascaded matching strategies. This type of method enhances geometric discrimination capabilities through aspect ratio changes or pseudo-depth information, and has certain advantages in scenarios with significant target pose changes. However, when the target shape is relatively regular, the pose change is not significant, or the target size is small and the appearance is highly consistent, the discrimination ability based on aspect ratio or pseudo-depth is limited, and it is still difficult to effectively distinguish dense targets, thus limiting the stability of the association.

[0006] In summary, existing multi-target tracking technologies still face the following challenges in scenarios with weak textures, highly similar target appearances, and dense spatial distribution: First, during the detection phase, multiple targets may stick together in the same connected region, affecting the accuracy of target initialization; Second, when relying solely on motion prediction of a single path or simple geometric metrics for association, it is difficult to maintain stable identity consistency in complex motion and high-density scenes. To address this, a detection and tracking algorithm and application system for weakly textured multi-target targets using a monocular camera are proposed. Summary of the Invention

[0007] In view of this, the present invention provides a detection and tracking algorithm and application system for weakly textured multi-target targets under a monocular camera, so as to solve or alleviate the technical problems existing in the prior art, and at least provide a beneficial option.

[0008] The technical solution of the present invention is implemented as follows: a detection and tracking algorithm for multiple targets with weak texture under a monocular camera, comprising a detection stage and a tracking stage executed sequentially; The detection stage includes a sticky target separation step, which models the distribution of foreground pixels in the candidate connected region by constructing a Gaussian mixture model to achieve sticky target separation. The tracking stage adopts a dual-path prediction-driven mechanism, which executes sparse optical flow prediction and multi-constraint Kalman filter prediction in parallel. The sparse optical flow prediction drift is corrected by comparing the geometric deviation between the two, and the prediction model parameters are updated based on the corrected observation results, thereby outputting the multi-target tracking results of the current frame.

[0009] The detection stages include, in sequence: S1. Image Acquisition and Detection Region Setting: Acquire the original image and preset the target motion detection region, then crop the detection region image; S2. Image preprocessing and target saliency: The image of the detection region is processed into a binary image with a black background and a white target; S3. Connected region analysis and candidate region extraction: Detect connected regions in a binarized image and filter to obtain a set of target candidate regions; S4. Separation of Adhesive Targets: Construct a Gaussian mixture model for each candidate connected region of a target, and divide the adhesive regions based on the Gaussian component distribution of the model. S5. Target Information Output: Calculate the structured information of each target and integrate and output the multi-target detection results.

[0010] The tracking phase includes, in sequence: T1. Multimodal input and preprocessing: Obtain the multi-target detection results of the current frame and the tracking result set of the previous frame, and use the live target tracking box of the previous frame as the prediction geometric reference for the current frame; T2, Sparse Optical Flow Prediction: Select sampling points in the tracking box to perform sparse optical flow tracking, and correct optical flow prediction drift based on correlation index; T3. Multi-constraint Kalman filter motion modeling: Construct the target motion model, set the state vector and observation vector of the Kalman filter, and introduce kinematic constraints and geometric correlation constraints to adjust the motion model; T4. Initialization and Dynamic Matching: A phased initialization strategy is adopted, and a cost matrix is ​​constructed through the Hungarian algorithm to realize the matching and updating between the prediction results and the detection results. T5. Dual prediction correction and feedback update: Two predictions are executed in parallel, with sparse optical flow prediction as the main localization basis and Kalman filter prediction as the reference benchmark. The prediction model parameters are updated after correcting optical flow drift.

[0011] The present invention also provides a detection and tracking system for multiple targets with weak texture under a monocular camera, including a detection module and a tracking module; The detection module includes an adhesion separation unit, which is configured to separate adhered targets by constructing a Gaussian mixture model to model the pixel distribution within the candidate connected region; the tracking module includes a dual prediction correction unit, which is configured to perform sparse optical flow prediction and multi-constraint Kalman filter prediction in parallel, and achieve optical flow drift correction and model parameter update by comparing the geometric deviation of the two.

[0012] Further preferably, the detection module also includes an image acquisition and region setting unit, a preprocessing and saliency unit, a connected region analysis unit, and an information output unit arranged sequentially according to the processing flow; each unit is configured to: acquire the original image and crop it to obtain the detection region image, process the cropped image into a binary image of black background and white target, filter to obtain a set of target candidate regions, divide the adhesion regions, and integrate and output the multi-target detection results.

[0013] More preferably, the tracking module further includes an input preprocessing unit, a sparse optical flow prediction unit, a Kalman filter modeling unit, and an initialization and matching unit arranged sequentially according to the processing flow; each unit is configured to: determine the prediction geometric reference of the current frame, correct the optical flow prediction drift, construct and adjust the target motion model, and realize the matching and updating of prediction results and detection results.

[0014] More preferably, the original image acquired by the image acquisition and region setting unit is a color or grayscale image, and the preset detection region is a rectangular or polygonal region.

[0015] More preferably, the state vector set by the Kalman filter modeling unit includes the center coordinates, aspect ratio, height, and corresponding rate of change, and the observation vector includes the observed values ​​of the center coordinates, aspect ratio, and height.

[0016] More preferably, the sparse optical flow prediction unit selects four geometrically symmetrical sampling points, and the correlation index includes overlap and scale change rate.

[0017] More preferably, in the phased initialization process of the initialization and matching unit, the transition phase is within the first three frames, the stable tracking period begins from the fourth frame, and the cost matrix is ​​constructed based on the geometric similarity between the optical flow prediction box and the detection box.

[0018] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: I. This invention introduces a method for separating sticky targets based on Gaussian mixture model in the detection stage. It performs pixel-level modeling of sticky or densely clustered target regions and achieves adaptive decomposition of single connected regions by multi-component Gaussian fitting of foreground pixel coordinates. This effectively distinguishes multiple similar targets whose detection boxes merge due to spatial proximity or local occlusion, thereby significantly reducing the false detection and missed detection rates in dense scenes and providing more accurate and reliable initial detection results for subsequent multi-target tracking.

[0019] Second, in the tracking stage, this invention arranges multiple uniform sampling points in the target area and performs sparse optical flow tracking. It uses multi-point motion information to characterize the local motion consistency of the target, and combines geometric constraints such as overlap and scale change to suppress optical flow prediction drift. This enables stable motion prediction to be maintained even under conditions of rapid movement, deformation or short-term occlusion, effectively enhancing the reliability of inter-frame target association and reducing the risk of identity switching.

[0020] Third, this invention constructs a dual-path prediction model that integrates sparse optical flow prediction and multi-constraint Kalman filtering. Sparse optical flow prediction is used to provide a fast response to nonlinear motion, while Kalman filtering is used to achieve state smoothing and motion trend constraints. The reliability of optical flow prediction is judged by the geometric deviation between the two prediction results, forming a closed-loop correction mechanism. This significantly improves the stability and prediction accuracy of target state estimation while ensuring real-time performance.

[0021] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the detection stage of the present invention; Figure 2 This is a flowchart of the tracking phase steps of the present invention. Detailed Implementation

[0024] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0025] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] like Figure 1-2 As shown, this embodiment of the invention provides a detection and tracking algorithm for weakly textured multi-target targets under a monocular camera, including a detection phase and a tracking phase executed sequentially. The detection phase includes a sticky target separation step, which models the distribution of foreground pixels in candidate connected regions by constructing a Gaussian mixture model to achieve sticky target separation. The tracking phase adopts a dual-path prediction-driven mechanism, which executes sparse optical flow prediction and multi-constraint Kalman filter prediction in parallel. The sparse optical flow prediction drift is corrected by comparing the geometric deviation between the two, and the prediction model parameters are updated based on the corrected observation results, thereby outputting the multi-target tracking results of the current frame.

[0027] The testing phases include, in sequence: S1. Image Acquisition and Detection Area Setting The current frame's original image is acquired using a fixed-mount monocular camera. The monocular camera can select appropriate resolution and frame rate parameters based on the application scenario. The original image can be a color image or a grayscale image. To reduce background interference and improve subsequent processing efficiency, a motion detection area for the target is preset according to the specific application scenario. The detection area is a rectangular or polygonal region in the image coordinate system. For example, in an industrial conveyor belt monitoring scenario, the conveyor belt coverage area can be preset as a rectangular detection area; in an indoor corridor scenario, the corridor passage range can be preset as a polygonal detection area.

[0028] The acquired original image is cropped to retain only the image content within the preset detection area as input for subsequent processing (i.e., the detection area image), while invalid background areas outside the detection area are removed, reducing the computational load of subsequent preprocessing and analysis.

[0029] S2, Image Preprocessing and Target Satinization Since weakly textured targets have gradual grayscale changes and lack feature information, it is difficult to directly identify the target. Therefore, preprocessing is required to achieve a significant distinction between the target and the background. Specifically, the detection area image obtained in S1 is first converted into a grayscale image. Then, the grayscale image is subjected to threshold segmentation processing. The image is divided into foreground (target) and background by a preset grayscale threshold to obtain an initial binarized image.

[0030] The initial binarized image undergoes foreground and background polarity adjustment, so that the target area appears as white pixels (grayscale value 255) and the background area appears as black pixels (grayscale value 0), ultimately obtaining a binarized image of a black background with a white target. The thresholding segmentation here can employ an adaptive thresholding algorithm (OTSU algorithm) to adapt to image processing needs under different lighting conditions, ensuring effective separation of the target and background.

[0031] S3. Connectivity Analysis and Candidate Region Extraction Connectivity detection is performed on the binary image of black background and white target obtained by S2. The 8-neighborhood connectivity judgment rule is used to identify the connected regions formed by all white pixels in the image. Each connected region corresponds to a potential target or multiple targets that are attached together. For each detected connected region, its geometric feature information is calculated. The geometric feature information includes, but is not limited to, the area of ​​the connected region, the coordinates of the circumscribed rectangle (x / y coordinates of the upper left corner, width / height), and the set of region contour points.

[0032] Based on preset area thresholds or geometric constraints, all connected regions are filtered to remove noise interference regions that are too small and background interference regions that are too large, ultimately obtaining a set of target candidate regions. For example, in an indoor people tracking scenario, the area threshold range for connected regions can be preset to 50-5000 pixels to filter out candidate regions that conform to the human body size range.

[0033] S4, Separation of Adhesive Targets In weakly textured multi-target images, overlap and occlusion can easily form connected regions. Directly classifying these as single targets can lead to detection errors. This step constructs a Gaussian mixture model to model the pixel distribution within candidate connected regions, achieving accurate separation of connected targets. The specific implementation process is as follows: S41. Pixel Extraction: For each target candidate connected region obtained in S3, extract the coordinate set (xi, yi) of all white foreground pixels in the region, where i is the pixel index. S42. Gaussian Mixture Model Construction: Based on the extracted set of pixel coordinates, a Gaussian Mixture Model (GMM) is constructed. This model is used to describe the pixel distribution characteristics within connected regions. The pixel distribution of adhered targets usually presents multiple cluster centers, corresponding to multiple Gaussian components of the Gaussian Mixture Model. S43. Model parameter estimation: Determine the number of Gaussian components in the Gaussian mixture model according to the preset model selection rules, and use the maximum likelihood estimation method to iteratively estimate the mean, variance, weight and other parameters of each Gaussian component until the model converges. S44. Adhesive Region Division: Based on the pixel distribution range corresponding to each Gaussian component, a single connected region is divided into multiple sub-regions. Each sub-region corresponds to an independent weak texture target, thereby achieving accurate separation of adhesive targets.

[0034] S5, Target Information Output For each independent target obtained by S4 separation, calculate its structured information: Center point coordinates: obtained by calculating the centroid of the pixels in the target area, that is, the average of the x-coordinates and the average of the y-coordinates of all pixels; Outer bounding box parameters: including the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width, and the height of the outer bounding box; Outer contour point set: The outer contour points of the target are extracted using the Canny edge detection algorithm to form a contour coordinate sequence.

[0035] The center point coordinates, bounding box parameters, and outer contour point set of each target are integrated into the multi-target detection result of the current frame and output as a data matrix or JSON format to provide input data for the subsequent tracking stage.

[0036] The implementation of the tracking phase specifically includes five steps, T1 to T5: T1, Multimodal Input and Preprocessing The system acquires the multi-target detection results of the current video frame (i.e., the structured information output by S5, including the bounding box coordinates and contour information of each target), and simultaneously inputs the multi-target tracking result set (Tracklets) of the previous frame. The tracking result set includes the historical tracking box coordinates, motion state parameters, tracking IDs, and other information of each surviving target.

[0037] Using the tracking bounding box positions of all surviving targets extracted from the tracking result set of the previous frame as the geometric reference for target position prediction in the current frame, the target range for tracking in the current frame is clearly defined, the tracking search area is reduced, and tracking efficiency is improved.

[0038] T2, sparse optical flow prediction Sparse optical flow prediction is suitable for tracking fast-moving targets, but it is susceptible to drift due to weak texture features. This step improves prediction accuracy through reasonable feature point selection and drift correction: Feature point sampling: On each target tracking box in the previous frame, four sampling points (the midpoints of the four sides of the tracking box) that are evenly distributed at the geometric center of the tracking box are selected as the feature starting points for sparse optical flow tracking. Selecting symmetrically distributed sampling points can improve the stability of optical flow tracking and avoid tracking deviation caused by the concentration of feature points. Sparse optical flow calculation: The Lucas-Kanade sparse optical flow algorithm is used to track the sampling points of all targets and obtain the corresponding projection position of each sampling point in the current frame; Drift correction: Calculate the optical flow prediction bounding box of the current frame based on the projection position of the sampling points, and then calculate the correlation index between the prediction bounding box and the tracking box of the previous frame, including the overlap ratio IOU and the scale change rate. If the correlation index is lower than the preset threshold, it is determined that there is optical flow drift. Drift correction is achieved by smoothing the projection position of the sampling points, and finally the optical flow prediction result of multiple targets in the current frame is obtained.

[0039] T3, Multi-constraint Kalman filter motion modeling Kalman filtering has good motion state prediction and noise suppression capabilities. This step constructs a multi-constraint Kalman filter motion model to provide a stable prediction benchmark for tracking. A linear state-space model of the target motion is constructed. The state vector of the Kalman filter is set as [center coordinates (x, y), aspect ratio r, height h, corresponding rate of change (vx, vy, vr, vh)], where (x, y) are the image coordinates of the target center point, r is the aspect ratio of the target bounding box, h is the height of the bounding box, vx and vy are the motion rates of the center point in the x and y directions, vr is the rate of change of the aspect ratio, and vh is the rate of change of the height. The observation vector is set as [center coordinate observation (x0, y0), aspect ratio observation r0, height observation h0], and the observation values ​​are derived from the multi-target detection results of the current frame. To prevent the filter from diverging due to detection noise or sudden target motion, threshold limits for target coordinate position and motion speed are added to the filter. For example, the threshold range for vx is set to -5 to 5 pixels / frame to avoid velocity prediction deviations caused by abnormal detection values. Correlation thresholds between coordinate position and target bounding box width and height are added to dynamically adjust the covariance matrix of the Kalman filter by judging the rationality of changes in target bounding box size. Based on the set state vector, observation vector, and multiple constraint mechanisms, the target motion model is stably updated through the prediction-update iterative process of Kalman filtering, and the Kalman filter prediction results are output, including the predicted position, size, and motion speed of the target.

[0040] T4. Initialization and Dynamic Matching A phased initialization strategy is adopted to ensure a smooth start to tracking, and dynamic matching is used to achieve accurate correlation between prediction results and detection results. Cold start phase (first frame of video): At this time, there are no historical tracking results. The Kalman filter is initialized with a velocity vector of 0, and a unique tracking ID is assigned to each detected target output by S5 to establish an independent tracker. Tracking transition phase (within the first three frames of video): At this time, the tracker has not yet fully converged and mainly relies on the prediction results of the sparse optical flow method for initial tracking. Dynamic matching: During the transition phase and subsequent tracking, a cost matrix is ​​constructed using the Hungarian algorithm to match and update the sparse optical flow prediction results with the current frame detection results. The cost matrix is ​​constructed based on the geometric similarity (such as the overlap ratio IOU) between the optical flow prediction box and the detection box. The higher the similarity, the lower the cost. The minimum matching cost of the cost matrix is ​​solved by the Hungarian algorithm to achieve a one-to-one correspondence between the predicted target and the detected target, thus completing the state update of the tracker.

[0041] T5, Dual Prediction Correction and Feedback Update Starting from the fourth frame of the video, a stable tracking period is entered. A dual-path prediction-driven mechanism is adopted, combining the speed of sparse optical flow prediction with the stability of Kalman filter prediction to achieve accurate tracking and closed-loop feedback. The system simultaneously runs the sparse optical flow prediction path and the Kalman filter prediction path to generate two sets of candidate prediction boxes. The prediction results of the sparse optical flow method are given priority as the main localization basis because they are more adaptable to the nonlinear motion of the target. Meanwhile, the prediction results of the Kalman filter are used as a "reference benchmark". The geometric deviation (e.g., Euclidean distance) between the sparse optical flow prediction box and the Kalman filter prediction box is calculated. This step sets a preset reasonable threshold range to judge the reliability of the optical flow prediction. If the deviation exceeds this threshold range, it is determined to be optical flow drift. At this point, the Kalman prediction value is used to correct the sparse optical flow prediction result, effectively solving the problems of insufficient generalization of existing deep learning-based tracking methods for untrained targets and weak identity differentiation when targets have similar appearances. It also avoids matching ambiguity in dense target scenes, ensuring the accuracy of target localization. Based on the final observations after dynamic matching, i.e. the corrected target position and size information, the distribution positions of feature points in sparse optical flow tracking and the state parameters of the Kalman filter are updated in reverse (the covariance matrix is ​​adjusted). The system integrates the tracking IDs, corrected center point coordinates, bounding box parameters, and motion state parameters of each target to output the multi-target tracking result for the current frame, completing the single-frame tracking process. Through continuous association of tracking IDs, it effectively solves the problems of ambiguous and incorrect identity matching after occlusion in scenes with dense targets and close spatial distances. The detection and tracking system for weakly textured multi-target targets using a monocular camera of the present invention has the following specific configuration and functions: The detection module executes steps S1-S5 of the aforementioned detection phase. Through a structured workflow, it achieves accurate detection and separation of weakly textured multi-target objects, providing reliable initial target information for subsequent tracking and avoiding amplification of identity matching issues at the tracking end due to detection errors. It includes an adhesion separation unit, and, sequentially arranged in the processing flow, an image acquisition and region setting unit, a preprocessing and saliency unit, a connected component analysis unit, and an information output unit. The configuration of each unit is as follows: The image acquisition and region setting unit acquires the original image (color or grayscale image) of the current frame through a fixedly installed monocular camera. Based on the application scenario, it presets a rectangular or polygonal target motion detection region in the image coordinate system. The region is then cropped to preserve the image content within the detection region (i.e., the detection region image) and output to the preprocessing and saliency unit. This unit reduces background interference and lowers the processing pressure in dense target scenes by limiting the detection range, laying the foundation for subsequent accurate detection. The preprocessing and saliency unit receives the cropped image output from the image acquisition and region setting unit and converts it into a grayscale image. It then performs adaptive thresholding (such as the OTSU algorithm) on the grayscale image to obtain an initial binarized image. Finally, it generates a binarized image of a black-background white target by adjusting the foreground and background polarities, achieving significant distinction between the target and the background. The binarized image is then output to the connected component analysis unit. This unit addresses the difficulty in detecting weakly textured targets due to a lack of features by enhancing the saliency of these targets. The connected region analysis unit receives the binarized image output by the preprocessing and saliency unit, and uses the 8-neighborhood connectivity rule to detect white pixel connected regions in the image; it calculates the area, bounding rectangle, and contour point set of each connected region, and filters connected regions according to preset thresholds or geometric constraints, removing interference regions such as dust and shadows to obtain a set of target candidate regions, which is then output to the adhesion separation unit. This unit initially distinguishes between targets and interference through geometric feature filtering, reducing redundancy in subsequent processing; The adhesion separation unit receives the target candidate connected regions output by the connected region analysis unit, extracts the set of foreground pixel coordinates within each candidate connected region, constructs a Gaussian mixture model based on the pixel coordinate set, determines the number of Gaussian components (corresponding to the number of adhered targets) according to the Bayesian Information Criterion (BIC), and estimates the model's mean, variance, weights, and other parameters using the maximum likelihood estimation method. Based on the pixel distribution range corresponding to each Gaussian component, the adhered connected regions are divided into multiple independent sub-regions (corresponding to independent targets), and output to the information output unit. This unit achieves adhesion target separation through pixel distribution modeling, solving the detection error problem caused by overlap in scenes with dense targets. The information output unit receives the independent target regions output by the adhesion separation unit, calculates the center point coordinates, bounding box parameters, and outer contour point set of each target, integrates the above structured information into the multi-target detection result of the current frame, and outputs it to the input preprocessing unit of the tracking module in a preset data format. The structured information output by this unit provides the tracking module with an accurate initial target state, avoiding tracking identity confusion caused by ambiguous initial information. The tracking module executes steps T1-T5 of the aforementioned tracking phase. Through a dual-path prediction and closed-loop correction mechanism, it achieves stable tracking of multiple targets with weak textures, specifically addressing issues such as insufficient generalization of untrained targets, weak differentiation of similar targets, and poor representation of morphologically stable targets. It includes a dual-prediction correction unit, and, sequentially arranged in the processing flow, an input preprocessing unit, a sparse optical flow prediction unit, a Kalman filter modeling unit, and an initialization and matching unit. The configuration of each unit is as follows: The input preprocessing unit obtains the current frame multi-target detection results output by the detection module's information output unit, and simultaneously acquires the multi-target tracking result set (Tracklets, including historical tracking box coordinates, motion state, tracking ID, etc.) from the previous frame. It extracts the tracking box positions of all surviving targets from the previous frame's tracking result set, using them as the geometric reference for target position prediction in the current frame. This geometric reference information, along with the current frame's detection results, is output to the sparse optical flow prediction unit and the Kalman filter modeling unit. This unit aligns multimodal information to provide a unified reference for subsequent dual-path prediction, ensuring tracking continuity. The sparse optical flow prediction unit receives geometric reference information (the position of the tracking box in the previous frame) from the input preprocessing unit. It selects four geometrically symmetrical uniform sampling points (e.g., the midpoints of the four sides) on each target tracking box in the previous frame. The Lucas-Kanade algorithm is used to perform sparse optical flow tracking on these sampling points to obtain their projection positions in the current frame. Correlation indices such as the overlap and scale change rate between the predicted optical flow bounding box and the tracking box in the previous frame are calculated. Drift correction is applied to the optical flow prediction results based on these correlation indices, and the optical flow prediction results are output to the initialization and matching unit and the dual prediction correction unit. This unit improves the stability of optical flow tracking for weakly textured targets by using four-point uniform sampling and geometric consistency constraints, avoiding drift caused by feature scarcity. The Kalman filter modeling unit receives the geometric reference information output from the input preprocessing unit and the detection results of the current frame, and constructs a state-space model of the target motion. The state vector is defined as [center coordinates (x, y), aspect ratio r, height h, and corresponding rate of change (vx, vy, vr, vh)], and the observation vector is [center coordinate observation (x0, y0), aspect ratio observation r0, and height observation h0]. Kinematic constraints (coordinate and velocity threshold limits) and geometric correlation constraints (coordinate and target bounding box size correlation thresholds) are introduced. Stable updates of the motion model are achieved by dynamically adjusting the covariance matrix, and the Kalman filter prediction results are output to the dual prediction and correction unit. This unit avoids filter divergence through a multi-constraint mechanism and designs multi-dimensional state vectors for morphologically stable targets, improving the representation capability of such targets. The initialization and matching unit executes a phased initialization process. In the cold start phase (first frame), the Kalman filter is initialized with a velocity vector of 0, a unique tracking ID is assigned to each detected target, and an independent tracker is established. In the transition phase (within the first three frames), the unit receives the optical flow prediction results output by the sparse optical flow prediction unit, constructs a cost matrix based on the geometric similarity between the optical flow prediction box and the detection box using the Hungarian algorithm, and updates the prediction results with the detection results of the current frame. The matching results are then output to the dual prediction correction unit. This unit ensures smooth tracking startup through phased initialization and uses geometric similarity matching to solve the problem of ambiguous identity association between similar targets. The dual prediction and correction unit receives the optical flow prediction results from the sparse optical flow prediction unit and the Kalman prediction results from the Kalman filter modeling unit in parallel during the stable tracking period (starting from the fourth frame). Using the sparse optical flow prediction results as the primary localization basis and the Kalman prediction results as a reference, it calculates the geometric deviation between the two and determines optical flow drift. If drift exists, it corrects using the Kalman prediction values. Based on the matching results (final observations) output by the initialization and matching units, it updates the feature point distribution positions of the sparse optical flow prediction unit and the state parameters of the Kalman filter modeling unit in reverse. It integrates the tracking IDs, corrected positions, and motion state information of each target and outputs the multi-target tracking results for the current frame.

[0042] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A detection and tracking algorithm for weakly textured multi-targets under monocular camera, characterized in that, The detection stage and the tracking stage are executed in sequence. The detection stage comprises a conglomeration target separation step, which separates the conglomeration target by modeling the distribution of foreground pixels in a candidate connected region through a Gaussian mixture model; and the tracking stage adopts a double-path prediction driving mechanism, and performs sparse optical flow prediction and multi-constraint Kalman filter prediction in parallel, corrects the drift of the sparse optical flow prediction by comparing the geometric deviation of the two, and updates the prediction model parameters based on the corrected observation results, thereby outputting the current frame multi-target tracking result.

2. The detection and tracking algorithm for weakly textured multi-targets under monocular camera according to claim 1, characterized in that, The detection stage comprises the following steps in sequence: S1, image acquisition and detection region setting: acquiring an original image and presetting a target motion detection region, and cropping to obtain a detection region image; S2, image preprocessing and target saliency: processing the detection region image into a black background and white target binary image; S3, connected region analysis and candidate region extraction: detecting the connected regions in the binary image, and screening to obtain a target candidate region set; S4, conglomeration target separation: constructing a Gaussian mixture model for each target candidate connected region, and dividing the conglomeration region based on the Gaussian component distribution of the model; S5, target information output: calculating the structured information of each target, and integrating and outputting the multi-target detection result.

3. The detection and tracking algorithm for weakly textured multi-targets under monocular camera according to claim 1, characterized in that, The tracking stage comprises the following steps in sequence: T1, multi-modal input and preprocessing: acquiring the current frame multi-target detection result and the last frame tracking result set, and taking the last frame survival target tracking box as the geometric reference for the current frame prediction; T2, sparse optical flow prediction: selecting sampling points of the tracking box to perform sparse optical flow tracking, and correcting the optical flow prediction drift based on the correlation index; T3, multi-constraint Kalman filter motion modeling: constructing a target motion model, setting the state vector and observation vector of the Kalman filter, and introducing kinematic constraints and geometric correlation constraints to adjust the motion model; T4, initialization and dynamic matching: adopting a phased initialization strategy, constructing a cost matrix through the Hungarian algorithm, and realizing the matching update between the prediction result and the detection result; T5, double prediction correction and feedback update: performing two kinds of prediction in parallel, taking the sparse optical flow prediction as the main positioning basis and the Kalman filter prediction as the reference benchmark, correcting the optical flow drift, and updating the prediction model parameters.

4. A monocular camera based detection and tracking system for weakly textured multi-targets, characterized in that, The detection module and the tracking module are included; The detection module comprises a conglomeration separation unit configured to separate the conglomeration target by modeling the pixel distribution in a candidate connected region through a Gaussian mixture model; and the tracking module comprises a double prediction correction unit configured to perform sparse optical flow prediction and multi-constraint Kalman filter prediction in parallel, and to realize optical flow drift correction and model parameter update by comparing the geometric deviation of the two.

5. The system of claim 4, wherein, The detection module further comprises an image acquisition and region setting unit, a preprocessing and saliency unit, a connected region analysis unit, and an information output unit arranged in sequence according to the processing flow; each unit is respectively configured to: acquire an original image and crop to obtain a detection region image, process the cropped image into a black background and white target binary image, screen to obtain a target candidate region set, divide the conglomeration region, and integrate and output the multi-target detection result.

6. The system of claim 4, wherein, The tracking module further comprises an input preprocessing unit, a sparse optical flow prediction unit, a Kalman filtering modeling unit and an initialization and matching unit arranged in sequence according to a processing flow; each unit is configured to determine a current frame prediction geometry reference, correct optical flow prediction drift, construct and adjust a target motion model, and realize matching update of a prediction result and a detection result.

7. The system of claim 5, wherein, The original image obtained by the image acquisition and region setting unit is a color or grayscale image, and the preset detection region is a rectangular or polygonal region.

8. The system of claim 6, wherein, The state vector set by the Kalman filtering modeling unit contains center coordinates, an aspect ratio, a height and corresponding change rates, and the observation vector contains observation values of the center coordinates, the aspect ratio and the height.

9. The system of claim 6, wherein, The sparse optical flow prediction unit selects four geometrically central symmetric sampling points, and the correlation index includes an overlap degree and a scale change rate.

10. The system of claim 6, wherein, In the phased initialization process of the initialization and matching unit, the transition phase is within the first three frames, and the stable tracking phase starts from the fourth frame, and the cost matrix is constructed based on the geometric similarity of the optical flow prediction frame and the detection frame.

Images